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PROGRAM 


Electronic Imaging & the Visual Arts 


*The Foremost European Electronic Imaging Events in the Visual Arts? 


Forum for Users, Suppliers & Researchers 


The key aim of this Event is to provide a forum for the user, supplier and scientific research 
communities to meet and exchange experiences, ideas and plans in the wide area of Culture & 
Technology. Participants receive up to date news on new EC and international arts computing & 
telecommunications initiatives as well as on Projects in the visual arts field, in archaeology and 
history. Working Groups and new Projects are promoted. Scientific and technical demonstrations 
are presented. 


Main Topics 


e 2D-3D Digital Image Acquisition 

* Leading Edge Applications: Galleries, Libraries, Education, Archaeological Sites, Museums & 
Historical Tours 

* Mediterranean Initiatives in Technology for Cultural Heritage: 
Synergy with European & International Programmes 


e [Integrated Digital Archives for Cultural Heritage and Contemporary Art 

* Management of Museums by using ICT Technology: Access, Guides, Documentation & Other 
Services 

* The Impact of New Mobile Communications on Cultural Heritage and Modern Arts Area 

e Semantic Webs 


* Human - Computer Interaction for Cultural Heritage Applications 
* Copyright Protection (Watermarking), Anti-Counterfeiting and Electronic Commerce 


* Culture and e-government 

e Activities and Programmes for e-learning 

* Applications to TV & Cinema 

e 3D Developments and Applications in the Cultural Heritage Area 
* Digital Theater 

* Cultural Tourism & Travel Applications 

e Art and Medicine 


WHO SHOULD ATTEND 


THE CULTURAL SECTOR: The Visual Arts Community including Museums, Libraries, 
Archaeological Sites, Educational Institutions, Commercial Galleries and Dealers, Auction Houses, 
Artists & Collectors 

THE HI-TECH INDUSTRY SECTOR: Multimedia Systems, Image Acquisition & Analysis, 
Data-bases, Display & Printing, ICT Industry, Telematics & Systems Manufacturing, On-line 
Information Services 

MEDIA & RELATED SECTORS: Publishing, Press, Film, Television, Photography, Printing, 
Advertising, Graphics Design, Consumer Media 

IMAGING SYSTEMS RESEARCHERS: Imaging Systems, 3-D Acquisition, Reconstruction & 
Representation Systems, Information Sciences 

TOURISM & TRAVEL SECTOR: Tourism Agencies & Operators, Travel Agencies 

THE GOVERNMENT SECTOR: Ministries of Culture and other Institutions involved in Cultural 
Heritage, Ministries of Industry, Education, Research and Science, Regional Governments 
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GTTI, CNIT, 

HITACHI LTD., 

GRUPPO SESA, 

CABEL, 

T.T. TECNOSISTEMI, 

FRATELLI ALINARI IDEA, 

CENTRICA, 

NEUMUS, 

VIDITRUST, 

VIRTUITALY, 

INN-3D, 

ICESP, 


CONSERVATORIO DI MUSICA LUIGI CHERUBINI, 


ENTE CASSA DI RISPARMIO DI FIRENZE. 


Co-Chairmen: Vito Cappellini, Enrico del Re — Florence University 
vito.cappellini@unifi.it enrico.delre@unifi.it 


ADVISORY BOARD 


President: James Hemsley, EVA Conferences International 
jrhemsley6@gmail.com 


Cristina Acidini, Accademia delle Arti del Disegno, Florence, Italy 
Paolo Blasi, University of Florence, Florence, Italy 
Enrico Bocci, Progetto Ricerca e Innovazione, Confindustria Florence, Florence, Italy 
Nikolay Borisov, ITMO University, Saint Petersburg, Russia 
Edoardo Calia, Istituto Superiore Mario Boella, Turin, Italy 
Andrea De Polo, Fratelli Alinari IDEA, Florence, Italy 
Takashi Hamazaki, DIS Project, Hitachi Ltd., Yokohama, Japan 


EVA Conferences Organiser: Monica Kaayk, acinom 5@hotmail.com 
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Antonella Castaldi, Roberto Caldelli, Alessandro Nozzoli, Alessandro Piva, Francesca Uccheddu, 
Luca Belloni, Riccardo Saldarelli, Giuliana Signorini, 

Silvia Capecchi, Paola Imposimato, Claudia Riva di Sanseverino 


EVA Organiser: Dipartimento di Ingegneria dell’ Informazione (DINFO) - 

University of Florence 

Via di Santa Marta, 3 — 50139 Firenze, Italy 
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PROGRAM 


1 - CONFERENCE 


Wednesday, 11 May: 14,15 — 19,10 
Thursday, 12 May: 9,00 — 17,55 


2- WORKSHOP 


Wednesday, 11 May: 9.00 — 13,00 


3- SPECIAL EVENT 


Wednesday, 11 May: 19,30 — 21,30 


4 - TECHNICAL EXHIBITION 
Thursday, 12 May: 10,00 — 17,00 


Venue: Hotel Pierre 
Via De' Lamberti, 5 
50123 Firenze 
Tel. +39 055 216218 Fax +39 055 2396573 
E-mail: pierre@remarhotel.com 
www.remarhotels.com 


ROOM A 


1 - CONFERENCE 


Wednesday, 11 May 


Chairmen: Vito Cappellini, University of Florence 
Enrico Del Re, University of Florence 


14,15 Opening: Luigi Dei, 
Rector of University of Florence 
Enrico Del Re, 
Director of Department of Information Engineering — 
University of Florence 
James Hemsley, 
EVA Conferences International 
Pier Luigi Rossi Ferrini, 
Vice-President Ente Cassa di Risparmio di Firenze 
Paolo Castellacci, 


President GRUPPO SESA 
15,15 Coffee Break 
ROOM A 
15,30 SESSION 1 - STRATEGIC ISSUES 


Chairman: Paolo Blasi, University of Florence, Florence, Italy 


"Innovative tools for the creation, reuse and Rossella Caffo 
retrieval of digital cultural content" Michael-Culture Association 
“IP Strategic Role" Daniela Mainini 
Anti-Counterfeiting Center 
Milan, Italy 
“High Quality Archive Project for Polo Takashi Hamazaki 
Museale Fiorentino: Exploitation Activities. DIS Project, Hitachi Ltd. 
Demonstrations and Exhibitions in Japan” Yokohama, Japan 
“eFoto Hamburg Lab” Christoph Wienberg‘, Horst Scholz? 


"IT and Project Management, 
Office of Hamburg’s CIO, 
Hamburg, Germany 
Information Technology and Digital Projects, 
Hamburg Ministry of Culture, 
Hamburg, Germany 
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"Museo Nazionale del Bargello" 


"Uffizi Virtual Experience. Da Giotto a 
Caravaggio" 


ROOM A 


17,30 
APPLICATIONS 
Chairman: 
St.Petersburg, Russia 


"Beauty as a Bridge Between Cultures" 


"Recent advances in spectral imaging 
for cultural heritage documentation 
and analysis" 


"Application of Big Data and Content 
Curation to Exploitation of Cultural 
Heritage" 


"Piezomusicolor. A natural Form of 
Technological Art" 


"The Use of Patient Specific Instrumentation 
during Total Knee Replacement Surgery" 


llaria Ciseri 
Museo Nazionale del Bargello, 
Florence, Italy 


Marco Cappellini", Paolo De Rocco’, Paolo 
Romoli', Marxiano Melotti”, Vito Cappellini?, 
'virtultaly, Florence, Italy 

?Bicocca University, Milan, Italy 

University of Florence, Florence, Italy 


SESSION 2 - NEW SCIENCE AND CULTURE DEVELOPMENTS & 


Konstantine Karczmarski, University Innovation Office, ITMO University, 


Giuseppe O. Longo 
Information Theory, 
University of Trieste, 
Trieste, Italy 


Jon Yngve Hardeberg, Sony George, 
The Norwegian Colour and Visual Computing 
Laboratory, NTNU, Gjøvik, Norway 


Konstantine Karczmarski 
Department of Innovations, 
ITMO University, 

Saint Petersburg, Russia 


Giuseppe Caglioti', Marco Marcon’, Tatiana 
Tchouvileva', Riccardo Della Ragione? 
'Politecnico di Milano, 

Milan, Italy 

"Magic Music s.a.s, 

Livorno, Italy 


Lawrence Camarda, Antonio D'Arienzo, 
Salvatore Morello, Michele D'Arienzo 
Orthopaedic and Traumatology Department, 
University of Palermo, Palermo, Italy 
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Thursday, 12 May 


ROOM A 


9,00 INTERNATIONAL FORUM ON “CULTURE & TECHNOLOGY 
Chairman: Vito Cappellini, University of Florence, Florence, Italy 


The structure of the FORUM is presented. 


Actual developments and perspectives are outlined: 


Speakers Include: 


Cooperation Groups 
Proposed Projects 

Funding Opportunities 
European Commission Plans 
(HORIZON 2020) 


- Cristina Acidini, President Accademia delle 
Arti del Disegno, Florence, Italy 

- Nikolay Borisov, President of Center of 
Design and Multimedia, ITMO University, Saint 
Petersburg, Russia 

- Edoardo Calia, Research Director, Istituto 
Superiore Mario Boella, Torino, Italy 

- Alberto Del Bimbo, Director Centro per la 
Comunicazione e l'Integrazione dei Media, 
Florence, Italy 

- Monica Carfagni, President Promo Design, 
Calenzano, Florence, Italy 

- Fabio Donato, University of Ferrara, Italian 
Representative in HORIZON 2020-SC6 
Committee 

- Paolo Zampini, Director of Conservatorio di 
Musica Luigi Cherubini, Florence, Italy 


11,00 Coffee Break 
11.15 SESSION 3 - NEW TECHNICAL DEVELOPMENTS & APPLICATIONS 
Chairman: Jon Yngve Hardeberg, The Norwegian Colour and Visual Computing 


Laboratory, NTNU, Gjøvik, Norway 


“Getting Past the Fear of Virtual Innovation” 


Eugenia Romanelli 
Italian Journalist 
Rome, Italy 


"Perception-Based Histogram Equalization 
for Tone Mapping Applications" 


"From Encyclopédie plates to visual 
knowledge for craftsmen: the 
project Visuoplanches" 


"Culture and Computer Science to 
Change Society's Behavior" 


"New Trends of 3D Technologies" 


“New 3D Scene Representation for FTV” 


13,15 Lunch Break 
ROOM A 


14,40 


S. Ploumis', Ronan Boitard', Mahsa T. 
Pourazad? ?,Panos Nasiopoulos" ? 

'Elect. and Computer Eng. Department, University 
of British Columbia, Vancouver, Canada 

?TELUS Communications Inc., Vancouver, Canada 
?The Inst. for Computing, Information and Cognitive 
Systems (ICICS), Vancouver, Canada 


Marcantonio Catelani', Maria Teresa Zanola?, 
Clara Vecchio? 

'University of Florence, 

Florence, Italy 

?Università Cattolica del Sacro Cuore, 

Milan, Italy 


Wilhelmina Ndapewa-Onyothi Nekoto!?, 
Hippolyte N'Sung-Nza Muyingi? 

Jürgen Sieck'? 

'University of Applied Sciences Berlin, 

Berlin, Germany 

?Namibia University of Science and Technology, 
Namibia 


Chiara Soffici', Francesca Uccheddu?, 
Francesco Falaschi?, Vito Cappellini’, 
'Master Thesis in Economy and Management, 
Florence University, Florence, Italy 

?DINFO - University of Florence, INN-3D, 
Florence, Italy 

*VARGROUP, Empoli, Florence, Italy 

"University of Florence, INN-3D, Florence, Italy 


Masayuki Tanimoto 
Nagoya Industrial Science Research Institute 
Nagoya, Japan 


SESSION 4 - MUSEUMS - VIRTUAL GALLERIES 
AND RELATED INITIATIVES 


Chairman: Andrea De Polo, Fratelli Alinari IDEA, Florence, Italy 


"Terahertz Advanced Research Techniques 
for Non-Invasive Analysis in Art Conservation 
(THz-ARTE)" 


M. Picollo', A. Aldrovandi", G. Bartolozzi‘, A. 
Casini’, C. Cucci', A. Doria?, K. Fukunaga‘, 
G.P. Gallerano?, E. Giovenale?, R. Olmi, M. 
Poggesi', L. Stefani! 


"Nello Carrara” Institute for Applied Physics of the 
Italian National Research Council (IFAC-CNR), 
Sesto Fiorentino, Florence, Italy 

?Opificio delle Pietre Dure (OPD), Florence, Italy 
*National Institute of Information and 
Communications Technology (NICT), Tokyo, Japan 


44 ENEA-Frascati, Frascati, Rome, Italy 


"The Secrets of the Work of Art. Mysteries 
concealed behind old Restorations and 
the Truth Revealed " 


"Cultural Assets, 'Extended' Museums and 
the Historic City: the Case Study of an App 
for Genoa" 


“3D Services at Museo Galileo" 


Sara Penco, 
Restorer ad Creator for the “SMARTICON Project”, 
Rome, Italy 


Valentina Fiore, Lauro Magnani, Sara Rulli 
Dept. D.I.R.A.A.S. 

Università degli Studi di Genova 

Genoa, Italy 


Luisa Barattin, Marco Berni, Elena Fani 
Museo Galileo - Institute and Museum of the History 
of Science, Florence, Italy 


16,00 Coffee Break 

ROOM A 

16,15 SESSION 5 - ACCESS TO THE CULTURE INFORMATION 
Chairman: | James Hemsley, EVA Conferences International, U.K . 


"Art in the Neuroscience Era. How the Brain 
Understands and Creates Art" 


"Disability & Digital Discussion: 
Global, EU & Italian Developments in 
the New Millennium" 


"Toviva Project: Documenting the Spanish 
Defense Towers along the Valencian Coast 
with a Comprehensive Digital Methodology 


"Rio's Digital Atelier Murat under 
Verrochio's Spell" 


Raffaella Folgieri', Ludovico Dei Cas?, 

Francesco Soave?, Claudio Lucchiari' 

'Dipartimento di Filosofia, Università degli Studi di Milano, 
Milan, Italy 

? CdL Scienze Biologiche, Università degli Studi di Milano, 
Milan, Italy 

3Msc Interactive Digital Media, 

Ravensbourne College of Design and Communication, 
London, U.K 


James Hemsley 

VASARI Research Centre, Birkbeck College, Universtity 
of London and EVA Conferences International, 

London, U.K. 


A. Pablo Rodríguez-Navarro', B. Giorgio Verdiani’, 
C. Teresa Gil Piqueras! 

'Departamento de Expresión Gráfica Arquitectónica 
Univesitat Politécnica de Valéncian 

Valencia, Spain 

?Dipartimento di Architettura, 

Università degli Studi di Firenze, 

Florence, Italy 


Heitor Luiz Murat de Meirelles Quintella, 
Stratimidia's Murat MAGIC 

(Maison d'Art Galerie et Imaginarium Contemporain) 
Atelier, 

Rio de Janeiro, Brasil 
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2 - WORKSHOP 


ROOMA 

WORKSHOP INNOVATION AND ENTERPRISE - INNOVAZIONE E IMPRESA 
(Italian Language) 

9,00 — 13,00 

Chairman: Enrico Bocci, Progetto Ricerca e Innovazione, Confindustria Firenze e 


Presidente Opera Medicea Laurenziana, Florence, Italy 


Technological requirements in the Cultural Heritage field are outlined and opportunities for 
Italian Enterprises and SME's working in the field, using new technologies, are presented. 
Regional and national applied research Programs in Italy are described. 

Activities by National Organizations and Firms working in the area of Telecommunications, 
Informatics, Environment and Infomobility are presented. 

Funding by European Commission is considered, with particular reference to multimedia 
and telematics for Cultural Heritage. Special consideration is given to the new EC Plan 
HORIZON 2020. 

Initiatives regarding the *know-how" transfer from Research Organizations to the Industrial 
Sector are described. 


Organizations and Companies present their activities and experiences. 


Invited Speakers: - Andrea Arnone, Pro-Rettore al Trasferimento Tecnologico e 
Presidente di CsaVRI, Università degli Studi di Firenze 
- Laura Castellani, Responsabile del Settore Infrastrutture e 
Tecnologie per lo Sviluppo della Società dell'Informazione, 
Regione Toscana 


Speakers include: | - Renzo Zampini, INFOCAMERE 
- Stefano Cinquini, TELECOM ITALIA 
- Paola Castellacci, VARGROUP 
- Daniele Corsini, CABEL 
- Riccardo Bruschi and Luca Bencini, T. T. Tecnosistemi, Prato 
- Francesca Gemma, Aracne editrice int.le, Roma 
- Gianpiero Alfarano, DESIGN CAMPUS, Calenzano, Firenze 
- Gianluca Vannuccini, Servizio Sviluppo Infrastrutture Tecnologiche, 
Comune di Firenze 
- Andrea del Re, Studio Legale Del Re — Sandrucci, Firenze 
- Franco Guidi, NEUMUS, Firenze 
- Carlo U. Quinterio Brentano, Philip Mazzei Association, Firenze 
- Alessandra Scappini, SINCRESIS, Empoli, Firenze 
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3- SPECIAL EVENT 


Wednesday, 11 May 19,30 — 21,30 


Visit to Opera Medicea Laurenziana 
Piazza San Lorenzo, 9, Florence 


(in cooperation with Antica Compagnia del Paiolo) 


4 - TECHNICAL EXHIBITION 


Thursday, 12 May: 10,00 — 17,00 


ROOMB 


For information on the Exhibition: 

CENTRICA S.r.l. 

Piazza della Madonna della Neve, 5 - 50122 Firenze - Italy 

Tel.: (+39) 055 24 66 802 — Fax: (+39) 055 20 09 785 

E-mail: info@evaflorence.it Web pages — http://www.evaflorence.it/home.php 


PROCEEDINGS 


SIRATEGIC ISSUES 


Vito Cappellini and Enrico Del Re (edited by), Electronic Imaging & the Visual Arts. Eva 2016 Florence. 
ISBN 978-88-6655-973-3 (print) ISBN 978-88-6655-974-0 (online) O 2016 Firenze University Press 


INNOVATIVE TOOLS FOR THE CREATION, REUSE AND RETRIEVAL 


OF DIGITAL CULTURAL CONTENT 


Rossella Caffo 


President of Michael — Culture Association 


<rosa.caffo@beniculturali.it> 


The Michael Culture Association will present 4 open-source tools developed within the AthenaPlus 
project, targeted mailnly to GLAMS curators who want to valorize their collections and cultural 
content in innovative ways. MOVIO: An open source and user-friendly tool for creating digital 
exhibitions in an easy, creative and fun way. It can be used by curators, students, schools, and 
whoever wants to create a digital exhibition. Urban Explore: a tool to create and manage mobile 
apps as touristic and cultural digital guides for collective or solo visits. As a complement to a 
spoken story (by a heritage expert, historian, city planner, or art professional), the mobile app offers 
an access to the audio-visual history or memory of a place, in an original and emotional way. 
CityQuest: a tool that allows cultural organisations to easily create a quest online, and publish it to a 
mobile app. SchoolTrip: a tool that allows students to create their own school journey. Through an 


online interface the teacher can set a couple of parameters defining the skeleton of the trip. 
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IP STRATEGIC ROLE 


Daniela Mainini 
President of Anti-Counterfeiting Center 


<presidenza@anticontraffazione.org> 


ABSTRACT 


IPR are crucial for the development of Industry and its ability to compete in a global economy. 

The establishment of IP Rigths is one of the principal means by which companies, creators and 
inventors generate returns on their investment in knowledge. 

In practical terms, through the granting of temporary exclusive rights, IP is directly linked to the 
production and distribution of new and authentic goods and services, from which all citizens 
benefit. 

The March 2015 European Council reaffirmed the importance of IP as a Key driver for growth and 
innovation and highlighted the need to fight against counterfeiting to enhance the EU’s industrial 
competitive scenario. 

To this end, the key to achieving these goals is an optimal and economically efficient IP 
“infrastructure “ which spans the legal recognition, utilization and balanced enforcement of all 
forms of IP rights. 

Price and not value: 

The Consumers often see only the price and not the value of what they have bought 

As Warren Buffett says: Price is what you pay. Value is what you get. 

The value of an authentic product is not just the price. Value includes legal employment, 
production practices, R&D, proprietary designs, marketing and product associated services. 

To this end, we are working to educate the consumer but one of the major tasks is to grow 
awareness among entrepreneurs. 

Italy should therefore dominate the ranking of the IPRI (International Patent Registration Index) 
and emerge as one of the countries that patents the vast majority of new products. It should be 
taking greater measures to protect intellectual property and manufacturing processes. This is clearly 
not the case. As the evidence from the European Patent Office (EPO) shows, Italy is behind 
Germany and France in the number of patents registered. 

ICT 

Information Communication Technology plays an essential role in protecting retail distribution and 
logistics systems and helps detect and exclude IP infringing products. 

In order to ensure product distribution chains are protected and not infiltrated by IP infringing 
products, technologies are being applied - such as the combination of BAR codes and the 
application of Radio frequency Identification (RFID) 

ICT has, of course, also allowed for an increase in direct delivery to the final consumer opening the 
way to the web and its huge supermarket, which is open 24/7. Equally, ICT solutions are also 
empowering consumers to fight against counterfeiting and piracy. 
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Of course: the basis for consumer empowerment is to be found in education _ e.g. Ethical 
traceability which tracks a product from farm to plate, is now widely used in the food sector, and 
aims at giving consumers an informed choice. 

The Italian Parliament is working to incentivise enterprises that are using ICT devices in their 
Supply Chains and finally the patent box has entered into force. 

However it is impossible to ignore the challenges presented by advancing technology . In the next 
twenty years we will be able to download a copy of art works which will be identical to the original. 
We only have to look 3D printing to understand that IP rights protection must keep step. 
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HIGH QUALITY ARCHIVE PROJECT 
FOR POLO MUSEALE FIORENTINO: 
EXPLOITATION ACTIVITIES 


Demonstrations and Exhibitions in Japan 


Takashi Hamazaki 
Director DIS Project, Hitachi, Ltd., Yokohama, Japan 
takashi.hamazaki@hitachi.com 


SUMMARY 
The “High Quality Archive Project for Polo Museale Fiorentino”, developed by MICC — 
University of Florence, Hitachi Ltd. and Centrica S.r.l., with supervision by Polo Museale 
Fiorentino, has been completed the first stage of digitization. Major activities in Japan 
under this project since the last presentation at EVA2015 so far are as follows. 


1. UVM2016 in Tokyo 

The “Uffizi Virtual Museum (UVM) 2016” was opened to the public from 13 February to 
13 March 2016 at Italian Cultural Institute in Tokyo as one of events for «150^ Anniversary 
of diplomatic relationship between Italy and Japan". UVM2016 was promoted by Italian 
Embassy in Japan, Italian Cultural Institute and Department of Information Engineering at 
the University of Florence. 

We have fully upgraded contents of the previous UVM, which was developed in 2011 and 
presented at EVA2012. First of all, the number of artworks covered this time was increased 
from 10 to 23. As for the quality of digital image presentation, we have utilized 4K devices to 
show them in better quality. Our unique high definition digital technology could make the 
best use of the latest devices, in a sense. 

The exhibition was composed of following 4 parts, i.e. “Life-Size Replica", “Digital 
Theater", *Masterpiece Navigation" and "Feel Uffizi". The combination of these various 
kinds of expressions could deeply move audiences more effectively. 


2. Permanent Exhibition (Hitachi DIS Showroom) 

Hitachi DIS Digital Museum, permanent Exhibition site at Yokohama, continues (since its 
opening in October 2012) to show UVM contents and other replica & digital data newly 
digitized to general public for free of charge. Its main purpose is to introduce Japanese and 
Western Culture including Italian Culture through our technology. 

Approximately 50 people visit the Museum on monthly basis with favorable reputations. 
Since this site is a real contact point with various type of audience, it should be very useful for 
us to develop a better plan. 


3. Miscellaneous Demonstrations 

To take advantage of the outcome from the project, we demonstrated our high-definition 
digital data at any opportunities, which include following events. 

The Lecture session for art-works of Renaissance, which is promoted by /talian Consulate 
General in Osaka, by Prof. Okada, Kyoto University, has taken place at the Museum of Kyoto 
in Nov. 2015. Masterpiece Navigation with digital data has been utilized to visualize his 
session effectively. 

The Exhibition of Leonardo Da Vinci by Kansai Telecasting Corporation has been taken 
place from Dec. 2015 to Feb. 2016 in Osaka. Masterpiece Navigation with digital data of 
Leonard Da Vince has been presented there. 
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eFoto Hamburg Lab 


Christoph Wienberg!, Horst Scholz 
'IT and Project Management Specialist at the Office of Hamburg's CIO. Hamburg, Germany 
Christoph.wienberg@fb.hamburg.de 


*Art Historian and Head of Information Technology and Digital Projects at the Hamburg 
Ministry of Culture, Hamburg, Germany 
horst.scholz@kb.hamburg.de 


Abstract 

Like many metropoles, Hamburg has vast collections of historic photos. Creating access for 
everyone to these collections is a task assigned to the Ministry of Culture by the Hamburg 
government as part of comprehensive programs covering digital / smart city strategies and the 
eCulture Agenda 2020’. 

To achieve digital access to the collections, several issues have to be addressed. Images must 
be digitalized and indexed. Legal issues have to be solved. A central digital platform to 
manage, store and publish images and metadata has to be built. But most important of all, 
applications to present those images in an entertaining and interactive, yet reputable way have 
to be created, to achieve public attention and participation. 

Hamburg has set up the eFoto? project to address this complex challenge. In eFoto, we will 
include image collections from several stakeholders, ranging from museum collections of 
photographic art (scanned and scientifically indexed) to local historic societies. Concepts and 
use cases have been developed in cooperation with Professor Jan-Christoph Meister’ and his 
team at the University of Hamburg, Faculty of the Humanities, using concepts of Niklas 
Luhmann. 

For a start, the topics of industrial history, national socialism and urban development / 
gentrification have been used to create historic city walks in the city quarter of Ottensen, 
using historical fotos from various sources and re-recorded historic statements. All Content 
has been researched and developed especially for eFoto by “Stadtteilarchiv Ottensen”*, a local 
historical society. Users will soon be guided through Ottensen by our mobile application. 
They will also be able add their own recordings of contemporary witness statements. A field 


test with pupils is scheduled for June 2016. A rollout to more partner organisations in 


* http://www.hamburg.de/kulturbehoerde/eculture 
? http://www.efoto-hamburg.de/ 

? http://icmeister.de 

“ http://stadtteilarchiv-ottensen.de 
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Hamburg, more features and a browser interface are planned for a second release scheduled 
for the end of 2016. 

Our implementation partner is HITeC e.V.°, a spinoff of the IT faculty of Hamburg University. 
A central multi-client repository is in the process of implementation, as well as mobile apps 
for iOS and Android for the city walks (fed from a central content management system). 
Current software modules include Liferay (a CMS capable of community management, free 
content object definitions and multi-language pages), Neo4J (a graph database used to store 
object relations) and RIAK, an object store used to store media data and compatible to 
Amazon S3. The metadata model is based on CIDOC and LIDO. 

We would like to invite interested parties to join our workshop. Have a look at the brand new 
beta release of the eFoto App and platform; discuss technical matters and (future) ideas of 
presentation and participation. Cooperation would be welcome in IT development (it's all 


open source), but also in the use of the eFoto platform, filling in your own local content. 


eFoto — Future model of partnership? 


Hamburg 


Partners in Hamburg Partners in Firenze Partners in ... 
Freie und Hansestadt Hamburg 


a http://www.hitec-hh.de/ 
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The Museo Nazionale del Bargello is among the most important museums in Florence. Here are 
displayed collections of Renaissance sculpture, with marble, wood and bronze masterworks by 
Donatello, Verrocchio, Desiderio da Settignano, Della Robbia, Michelangelo, Cellini, Giambologna 
and many others great artists. The museum includes also famous collections of decorative arts, 
coins and medals, weapons, textiles, paintings and one of the main collection of maiolicas in the 
world. The Bargello celebrated its 150th anniversary in 2015, since it is the first National Museum 
of Italy, estabilished in 1865 in the ancient Palazzo del Podestà: among the founding collections 
was the historic Grand Ducal Armoury, that includes the Medici and Della Rovere weapons and 
comprises over 2000 items. Every year the Museum organizes temporary exhibitions concerning 
sculpture and decorative arts, hosting also innovative projects that open towards advanced 
technologies. In 2015 has been presented “Mnemosyne — Smart Museum Project”, a sperimental 
project funded by the European Commission and Regione Toscana and realized by the MICC of the 
University of Florence: the project developed a system wich analyse visitor's behaviour through a 


multimedia touchscreeen collocated in the Donatello Room. 
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ABSTRACT 


This paper presents “Uffizi Virtual Experience - da Giotto a Caravaggio”, the first 
immersive and interactive virtual exhibition created in Italy, which brought to Milan 
the masterpieces of the Florence Uffizi Gallery with high-resolution digitized images. 
The exhibition, which was produced by Virtultaly, a Centrica startup spin off 
company, conveys a new approach to edutainment, where cultural recreation is 
achieved through an immersive educational and entertaining experience. 
Thanks to digital technologies and interactive systems, educational content and 
cultural enjoyment are integrated with emotional aspects to create a satisfying 
formative activity. 

The paper dwells upon the technological, sociological and cultural aspects of this 


new way of presenting and promoting cultural heritage in our society. 
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The perception of beauty is produced by an emotional catastrophe 
René Thom 


Abstract 


A naturalistic definition of beauty is proposed based on our immersion in a systemic 
context and in a vast ongoing co-evolutionary process: beauty exhibits itself and is 
recognized by us through the interaction and the reciprocal dynamical adaptation between 
ourselves and the environmental context. This system evolves in time and so does that 
adaptation, hence beauty has a historical character and depends on the physical, chemical 
and biological properties of the environment, on the psycho-biological characteristics of 
human beings, on their individual histories, and on the specific cultural and geographical 
contexts. 


INTRODUCTION 


The debate on the “two cultures” started in 1959 with the famous lecture of Charles 
P. Snow entitled precisely The Two Cultures. The opposition between scientific culture 
and humanistic culture has sometimes assumed overheated and almost grotesque tones. 
Actually the relationships between the so-called two cultures are very tight and their 
scopes and aims are closer than one can suppose. First, they meet in, and are produced 
by, human beings; and, second, they share the goal of seeking adequate and meaningful 
representations and models of the world and of man. Although they speak different 
languages and are driven the first one by tendentially objective and communicative 
goals and the other by subjective and expressive goals, finally both turn out to be 
intersubjective. The recognition of beauty is subjective, but the common nature and 
shared experiences of human beings make it intersubjective. One can therefore wonder 
whether it is justified any attempt to evaluate beauty in a tendentially objective way, 
going beyond the particular codes of interpretation and representation to reach the 
fundamental level of reality that mathematics and physics try to investigate. In this 
contribution I would like to put forward some considerations on beauty, in the hope that 
it turns out to be recognized as a bridge between the two cultures. 


AESTHETICS AND ETHICS 

I now come to the core of my contribution. I propose the following systemic and 
evolutionary definition of aesthetics, considered as a combination of sensitive abilities 
and activities, in particular in front of beautiful objects: 
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e Aesthetics is the subjective (but shared) perception of our bond with the 
environment in which we are immersed and of which we are part; such a bond is 
characterized by a deep and balanced dynamical harmony and displays itself in the 
recognition of beautiful objects through senses and emotions. 


Next I make a step forward by asserting that, in my naturalistic perspective, aesthetics is 
closely linked with ethics, for which I propose the following definition: 


e Ethics is the subjective and intersubjective capability of conceiving and carrying out 
actions that keep our bond with the environment healthy and balanced. Such actions 
maintain and increase beauty. 


Ethics and aesthetics are the two sides of the same coin since they arise from the strong 
systemic and evolutionary co-implication between species and environment and both are 
"mirror images" of that co-evolution within ourselves. 

Thus aesthetics is the (inter)subjective perception of the balanced (vital) immersion in 
the environment and ethics is the (inter)subjective perception of the necessary respect 
for, and suitable action within, the environment. As a consequence, ethics permits us to 
keep aesthetics and aesthetics functions as a guide in acting ethically. Both ethics and 
aesthetics are rooted in our evolutionary history, in fact in the co-evolution between the 
environment and ourselves. Such a primordial evolutionary connection concerns 
beautiful objects, whereas beautiful concepts, e. g. those that some find in mathematics, 
are much more recent philogenetically, hence recognizing their possible beauty is more 
difficult, involves rationality, is less spontaneous and requires a specific individual 
training. In the case of natural objects, a great part of such a training was carried out in 
our place by the past generations. [1] 

As can be noticed, in the proposed definitions there is a shift from the subjective 
position to he intersubjective one. Such a shift is justified by the common nature and 
history of human beings, from the fundamental physical and chemical components, 
through the shared genetic and biological roots, up to the common existential 
experiences. The electron charge, the gravitational force, the proton mass and so on: all 
of these characteristics have conditioned the rise and development of life on the earth 
and have been impressed upon us. This justifies what Vitruvius (80-23 B. C.) and later 
the alchemists asserted, that there is a strict correspondence between the macrocosm 
(the world) and the microcosm (man). 


BEAUTY IN MATHEMATICS AND PHYSICS 


The attempt to represent reality concerns in particular mathematics. But art and 
mathematics exhibit a deep difference: actually we live in a world of forms and colours 
that we perceive with our senses, and only secondarily we dwell in a world of concepts 
that we understand with our minds. To understand mathematics we need to master 
difficult techniques and tools that we can only acquire through a long and difficult 
training. Everybody can listen to a symphony or gaze at a painting, enjoying it at some 
level, whereas mathematics cannot be listened to or gazed at in the same way. The final 
test of an art work resides in the subjective (or better intersubjective) aesthetic pleasure 
that it gives, while, concerning mathematics, mathematician Morris Kline says: “The 
question whether mathematics possesses or not a kind of beauty can only be answered 
by those who have a culture in this domain. Unfortunately, to master mathematical ideas 
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one needs years of study and there is no via regia or king's highway that can shorten the 
process." 

For instance, very few laymen would be able to appreciate the fascination of the 
following amazingly simple expression (due to Euler) 


e™+1=0 


that connects five of the most important number in mathematics, i. e. 1, 0, i, e, and m. 
Many mathematicians consider this to be the most beautiful mathematical formula. But 
to appreciate its beauty one needs to be a mathematician, because we did not co-evolve 
with mathematics, but with the world, and our interaction with the world has been 
mediated more by the body than by the mind. For sure, mind, too, has its evolutionary 
history, but the abstraction process that has led to mathematics is very recent. The co- 
evolution between us and the world is much more ancient, and it has impressed in us the 
coordinates (images) of sunsets and trees and animals and human beings, not of 
mathematical expressions. 

The great mathematician Geoffrey H. Hardy was also a supporter of beauty. In his book 
A Mathematician's Apology he wrote: “The mathematician's patterns, like the painter's 
or the poet’s must be beautiful: the ideas, like the colours or the words must fit together 
in a harmonious way. Beauty is the first test: there is no permanent place in this world 
for ugly mathematics." [2] 

The prominent physicist Paul A. M. Dirac, Nobel laureate (1933), would take beauty 
both as a guide in finding meaningful results and as a criterion to test the validity of the 
formulae. He went as far as to write: “It is more important to have beauty in one's 
equations than to have hem fit experiment. It seems that if one is working from the point 
of view of getting beauty in one's equations, and if one has really a sound insight, one is 
on a sure line of progress. If there is not complete agreement between the results of 
one's work and experiment one should not allow oneself to be too discouraged, because 
the discrepancy may swell be due to minor features that are not properly taken into 
account and that will get cleared up with further developments of the theory." [3] 


CONCLUSION 


It is almost obvious to everybody that beauty holds sway in the realm of humanities. 
Although we are probably not able to define beauty, we recognize it when we meet it, 
both in nature and in art, literature, music. So it seems that beauty is of paramount 
importance in both the so-called two cultures, and can well be considered as a bridge 
between them, so reaffirming the unity of culture and, more deeply, of man. 
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EXTENDED ABSTRACT 


Cultural heritage documentation and analysis with imaging techniques has changed 
tremendously over the past few decades. After the transition from film cameras to digital 
technology, the next objective was to record scientifically valuable image datasets of the 
artifact. Different imaging devices have been used to fulfill this drive including spectral 
cameras, 3D imaging, reflectance transformation imaging, etc. 

There has been several efforts to capture accurate colour, shape, and spectral / material 
properties of cultural heritage objects. In this paper, we discuss the progress in research and 
development at The Norwegian Colour and Visual Computing Laboratory (Colourlab) 
devoted to acquisition and analysis of paintings using spectral imaging. Originated from the 
field of remote sensing, hyperspectral imaging technology has been explored in various 
sectors, including computer vision, medical imaging, and material classification. In the 
cultural heritage domain, spectral imaging has been used for precise scientific documentation, 
material identification, enhanced visualisation, virtual restoration etc. Many of the used 
spectral imaging systems were not initially developed for cultural heritage imaging, but taken 
from other application domains, there is several pre and post-processing steps need to be 
investigated and implemented to acquire quality image dataset [1, 2], which can be used to 
achieve the above-mentioned objectives. 

The Colourlab has engaged in several spectral image acquisition and analysis campaigns of 
cultural heritage paintings at museum environments and at imaging laboratories [3]. For 
practical reasons, the same image acquisition setup do not work the same way in all these 
conditions. This causes spectral and spatial distortions of several forms which affects the 
acquired images. In order to include appropriate correction methods in the processing 
workflow, we investigate the kind of distortions from imaging systems in different 
configurations and propose new correction techniques [4]. 

Pigment mapping or material identification is an interesting topic in the cultural heritage 
sector. The potential of hyperspectral imaging technology to capture reflectance spectra at 
every spatial point of the scanned region make this a promising non-invasive tool for the 
identification of pigments and its spatial distribution across the painting. Investigations has 
been made in this direction to classify pigments based on their spectral characteristics [5]. 
Different spectral classification and unmixing algorithms has been evaluated on a 
hyperspectral image dataset of Edvard Munch’s masterpiece ‘The Scream’ and the results are 
validated with an analytical study carried out before [3]. 

Crack detection for cultural heritage objects is another interesting area still to be 
explored by the spectral imaging community. The potential of hyperspectral datasets to 
process the data metrologically can be employed for crack detection in paintings. Although 
crack removal may be considered a way to improve the perceived image quality of the 
painting, detection and documentation of cracks in paintings are motivated by several reasons 
such as understanding the degradation/aging process, as an indication of the painting’s 
authenticity. The colourlab has taken a step further in this research to develop mathematical 
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tools for crack detection of from hyperspectral images [6]. Distance based spectral 
morphological methods are developed to detect cracks in paintings. The performance of the 
proposed methods were validated with a set of artificially constructed images and real 
hyperspectral images. 

Many of the spectral imaging systems are not just limited to the visible range, but 
extended over the spectral region - near IR (NIR) and short wave IR (SWIR). This wide span 
of spectral data at very closely spaced wavelengths allows visualizing the acquired images in 
several ways - the true colour image of the artifact under any illuminants and the false colour 
images. For false colour visualisation of the artifact, we employ different mathematical 
operators and dimensionality reduction algorithms to choose the images with maximum 
distinct information. This brings our many barely visible details in different layers of the 
painting and serves as a tool for museum experts and visitors [7]. 
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ABSTRACT 


Introduction. Different social groups have desire to consume cultural content, but their perception in 
many cases is so different that to meet multiple audiences! expectations we need to provide a variety of 
technologies, content layers and interfaces. 


l. The pain. Like in most of the media industry, the problem of finding the right audience for a 
particular piece of knowledge has not been solved without tremendous amounts of money poured into 
advertising. Which, in case of cultural heritage, can create only “blockbuster culture” leaving most of 
the heritage without slightest attention. At the same time most of non-professional viewers are also left 
without access to culture because channels of communication between them and the cultural heritage 
are either not established or jammed by “blockbusters”. 


2. Current attempts of solving this problem. Even though there are certain attempts of 
connecting diverse audience, using big data, for example to drive people from the web to multimedia 
exhibitions (so far the most successful example of technology-driven cultural experience) , the results 
are far from desired. Reasons are: not enough relevant data cross-sections, and only analysis without 
attempts of synthesis of content. And content is not only “raw”, digitized artifacts, it is essentially 
stories, narratives. And each audience needs it's own story. 


3i Difficulties in production of digital content without sufficient data about audiences. Content 
that really draws attention is expensive to produce, hence production represents very high risks 
associated with the lack of knowledge about potential audiences and it's locations. Available 
"traditional" metrics does not provide enough data. Hence content is not produced. 


4. V-Memes, or “cultural codes” of Don Beck's Spiral Dynamics Integral. Building another layer 
of data about audience and artifacts and unlocking its potential to establish communication between 
cultural heritage and diverse audiences. 


Sa Application of Big Data in synthesis of narratives, storytelling and content distribution. 
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Abstract — PiezoMusiColor (PMC) is a vibro-chromo-phonoscope, allowing to disclose the 
visual and vibro-tactile dimensions of sounds, music and voice. Its applications range from: 
education to entertainment and social inclusion for impaired people. In education it is fruitfully 
applied in Learning by playing where some relevant physical laws can be explained in a 
interactive approach, while in the entertainment context PMC offers a full immersive multi- 
sensorial (acoustic, visual and tactile) approach able to hypnotically capture the user's attention. 
Concerning social inclusion, PMC allows people affected by hearing impairment or cognitive 
deficiencies, to correlate sound with touch and vision increasing their fruition capabilities. 


INTRODUCTION 


In the history of culture very few phenomena have occurred capable to attract and to intrigue 
not only artists and musicians but also humanists, scientists and laypersons. Listening to 
music while perceiving its patterns and colours in an immersive experience is one of these 
phenomena. 

This inspiration goes back to Isaac Newton, who conceived a parallelism between sound 
waves and light waves " ...May not the harmony and the discord of colors arise from the 
proportions of the vibrations propagated through the fibers of the optic nerve into the brain, 
as the harmony and discord of sounds arise from the proportion of the vibrations in the air?" 
(Isaac Newton, 1666) [1]. Nowadays the proposed solutions usually focus on computer 
graphics programs that are commonly based on algorithms able to convert sound into colours 
and random patterns “following” the music itself. The systems usually adopted for this 
application are categorized as "Visual Music"[2]. Historically the cathode ray tube made 
possible the oscilloscope, an early electronic device that can produce images that are easily 
associated with sounds from microphones. 


Figure 1 Some Lissajous figures obtained on cathode ray tubes "dancing" with a 
properly processed audio signal 


The modern Laser lighting display displays wave patterns produced by similar circuitry. The 


imagery used to represent audio in digital audio workstations is largely based on familiar 
oscilloscope patterns. 
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The Animusic company has repeatedly demonstrated the use of computers to convert music 
— principally pop-rock based and composed as MIDI events — to animations. Graphic artist- 
designed virtual instruments which either play themselves or are played by virtual objects are 
all, along with the sounds, controlled by MIDI instructions[3 ]. 


Figure 2 Laser patterns obtained from audio inputs. 


However the results often exhibit repetitive, cold and predictable structures that are criticized 
by musicians, painters and art critics for the underlying synthetic approach. 

The Musicolor technology is not based on any software or handcrafted algorithm, it is a 
natural form of multimedia graphic-art delegating to the music of all genres — or in general, to 
any kind of sound, like the voice itself — the function to trigger and control a process 
producing in a real-time an analog self-portrait of sound. Activating a natural process the 
music manifests its second nature: an endless stream of patterns, with rainbow colours, 
rhythm, timbre and intensity of sounds. 

Unlike Painting and Sculpting — static arts defined in two and three space dimensions 
respectively — Music is an art defined in the time domain. In the early years of the last 
century, futurist painters disclosed the time dimension to the static canvas of the pictorial 
space[4][5]. In a manner complementary to Futurism, Musicolor discloses the chromatic and 
space dimensions to the dynamics of the times of sounds. 


THE PIEZOMUSICOLOR PARADIGM 


Musicolor technology was initially developed in collaboration with Góran Rámme from the 
University of Uppsala (Sweden) [6], then, in recent years, vibrating components has been 
implemented revealing the third nature of music — its tactile dimension —. These components 
convey the vibrational self-portrait of sounds to the hands (by vibrating actuators) and/or to 
other constituents of the body, depending on the final user. The audio frequency spectrum has 
been properly reshaped and shrink in order to fit to the tactile spectrum (which is almost an 
order of magnitude smaller in the frequency range with respect to the audio one). 

Among the vibrational (or piezo-) components of the resulting analog PiezoMusiColor vibro- 
chromo-phonoscope we also included "AcousticA", a sonic armchair designed by Magic 
Music s.a.s. for the multi-sensory stimulation. AcousticA has been created combining the 
traditional techniques of violinmaking with the most innovative technology in sound research. 
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It is not a lounger that uses the normal speakers or sub-woofers: it is a tool capable of 
bringing the sound frequencies into resonance with different parts of the human body. The 
result is a deep massage acting on all muscular bands and the nerve endings of the body. The 
AcousticA armchair is activated by music composed by Riccardo Della Ragione specifically 
for music therapy, using physical and medical criteria, rhythms and sound frequencies that 
give optimal stimulation on the human body. It is currently adopted in hospitals and 
rehabilitation facilities for the elderly and children with disabilities, and it is also adopted in 
wellness areas and protocols of complementary medicine with excellent results. As an 
alternative to the AcousticA armchair a vibro acoustic pillow, the "CuscinosonorO", realized 
by Magic Music s.a.s., can be adopted. it is syncronous with Musicolor and it can be 
activated by all music genres. 


A possible PiezoMusiColor Setup 


In Figure 4 it is possible to see a possible implementation of the PMC project where a 
properly processed audio signal is mechanically transduced into the vibration of a thin film 
whose density is locally changing accordingly to global and local material properties. 
Micrometrical variations in the film thickness act as a classical Fabry-Perot interferometer [7] 
selecting different wavelengths (different colors) for the reflected light.Figure 1 


Digital to Analog converter 
and real time audio filters 


sound to motion 
processor 


Figure 4 An implementation step of the MusiColor system: the audio signal is 
transformed into a PMC video and recorded in real time. 
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In particular it is possible to identify in the generated video sequences some components that 
are more related to deterministic and predictable behavior and some more chaotic components 
related to complex fluid dynamics. The overall effect is an harmonious mixture of these two 
components as can be seen in Figure 5. 


Figure 5 Two frames of a PMC realization, well defined periodic patterns superimposed 
on chaotic movements are well visible. 


APPLICATIVE SCENARIOS 


During the last years many different applicative scenarios were proposed and adopted for the 
PMC system, in particular we mention the following ones: 


Education 


The PMC makes possible an interactive teaching of fundamental concepts and principles of 
natural sciences — learning by playing. An analog transcoding of music and voice gives rise to 
a faithful self-portrait of music in the audio-visual and vibro-tactile dimensions of sound: a 
succession of visual forms, sometimes chaotic and unpredictable, as might be the eddies of a 
river or the color of the flame, dance synchronously with music and with vibrating vibro 
transducers. For teaching purposes, the joyful sensations produced by the synaesthetic 
perception of sound trigger curiosity about the wide variety of concepts and laws of nature 
upon which the operation of the PMC is based. 


Entertainment 


The colorful symphony of visual forms, obtainable by Musicolor without any artificial 
processing, enhances the emotional impact of music stimulating a synaesthetic tension aiming 
to correlate the sensory tactile channel with hearing and sight: a novel, hypnotic and 
immersive music experience. 

Furthermore PMC can be fruitfully adopted in an endless set of possible applications; e.g. 
people can "wear their voice" just printing a frame of the PMC on cloth (an example is visible 
in Figure 6). 


Social Inclusion 
With the increase of the lifetime expectancy the problem of social integration of persons with 


physical, cognitive, mental, sensory or emotional disability becomes more and more serious. 
According to our experience with persons affected by hearing loss or cognitive deficiency, the 
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PMC is a promising and efficient tool. In particular we deployed a specific protocol at the 
Don Gnocchi clinic on patients with a large mental impairment verifying, after a couple of 
weeks, an increase in their cognitive capabilities concerning multi-sensorial stimuli 
correlation. 


Figure 6 "Wear your voice" an example of a possible entertainment application. 


CONCLUSIONS 


In this paper we presented the PiezoMusiColor, a multifaceted and versatile system able to 
convert whichever kind of audio source into an immersive, multi-sensorial and synaesthetic 
experience. In a few years from its first realization we have been able to fruitfully integrate it 
into many applicative scenarios, ranging from entertainment, to a clinical aid for music 
therapy, from education to an aid for deaf impaired people and we think that, in the close 
future, it will find further challenging applications. 
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ABSTRACT 

During the past decade, total knee arthroplasty (TKA) has been markedly increased. In order 
to improve the position of implants during TKA and to reduce the risk of early failure of the 
procedure, new technology has been introduced and used by several surgeons. Specifically, three- 
dimensional imaging and custom manufacturing have enabled the development of patient-specific 
instrumentation (PSI) that are used as cutting jigs. These cutting blocks are specific to a patient’s 
knee anatomy and should help the surgeons to perform bone cuts, reducing the complexity of 
conventional alignment and sizing tools. 

With this technology, a pre-operative MRI or CT is performed on the patient's knee. Hip 
and ankle images are also obtained for the evaluation of the overall alignment of the limb. With a 
specific software program (Mimics, Materialise, Leuven, Belgium), manufacturing engineers turn 
two-dimensional CT or MRI images into three-dimensional representations of the knee and lower 
limb of the patient's anatomy. Using a specific software, the surgeon is then able to evaluate the 3D 
planning of the knee implant with the proposed bony resections, modifying as necessary bony 
resection of the tibia and femur. After the surgeon’s approval, custom cutting guide that fit on the 
patient's native anatomy is manufactured and then sent to the surgeon. During surgery, the PSI 
guides are used as slotted cutting guides or for accurate pin position for the use of standard 
manufacturer resection instrumentation. 

Several advantages have been advocated using PSI technology. At first, the surgical time is 
reduced because many steps of the surgery have been already performed such as pre-operative 
planning. Other advantage includes a decrease in the instrumentation trays optimizing the operative 


room time and reduction of the operating time. Despite several potential surgical advantages of 
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using PSI technology, there are no long-term implant survival data to support its use. In addition, 
few studies evaluated the effectiveness of this technology over conventional instrumentation, and 
results of post-operative alignment accuracy are conflicting. 

From September 2012 to December 2014, 22 patients were treated with TKR using PSI. CT 
images of the lower limb were used to obtain DICOM images. PSI were produced by Medacta 
(Castel San Pietro, Switzerland), and used in all patients for femoral and tibial bone resection. A 
post-operative CT performed in all patients treated, showed a suboptimal positioning of the tibial 


component and an optimal positioning of the femoral component. 
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GETTING PAST THE FEAR OF VIRTUAL 
INNOVATION 


by Eugenia Romanelli 


In 2010, a research by Finnish tech company Nokia revealed something 
astonishing: people check their phones at least 150 times a day. This means once 
every 6.5 minutes, counting up to 22 calls and 23 text messages only during the 
day. While this was probably perceived as a huge business opportunity for Nokia, 
it made many columnists and intellectuals worried, raising concerns about our 


relationship with technology and virtual forms of communication. 


If a similar research were to be done today, it would reveal much higher rates. 
As more and more people become more prone to accepting the digitalization of our 
lives, some still argue that in today's world human existence is nothing but an 
extension of technology. Starting in 2014, a platform called *Open Forum", as well 
as American managing consultant Brian Moran, have lead the way in trying to 
detox people from the digital world, to let help them enjoy what Moran called 
“here and now”. However, if one were to explore our understanding of what living 
in symbiosis with technology really entails, it would be easy to see that it is by no 
means anything new: it is the same theme recurring in the story of Ulysses facing 
the “Pillars of Hercules" — in other words, people's fear of technology is merely 
a new interpretation of the age old dilemma of human endeavor versus fear of 


pursuing knowledge. 


Many statistics show that by 2020 almost 75% of the world economy will be 
dependent on the internet. According to Cisco (one of the world leaders in 
networking, active in promoting the "Internet of Things" theory) at least 10 billion 
devices will be connected to the internet by 2018. Considering the incredible 


amount of business that is being conducted online, Sheryl Sandberg, COO of 
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Facebook, has expressed interest on how the world will look like when 10096 of 


the population will have access to the internet (as opposed to today's 39%). 


Not only is there factual evidence to say that most communities in the world 
now are structured around digital networks, but it is also generally safe to say that 
such networks allow us to develop more creativity, can improve our cognitive 
processes and facilitate exchange of information. Our digital reality is also a great 


platform for social change, as well as promotion of democratic values. 


A paper called “Growing a Digital Social Innovation Ecosystem for Europe", 
commissioned by the European Commission, was presented in February 2015 at a 
panel called “Shaping the Future of Digital Innovation in Europe". Written on 
behalf of the National Endowment for Science, Technology and the Arts (NESTA), 
the paper argued that one of the largest digital communities is focused on 
awareness networks and innovation and can provide important tools to promote 
integration and increase people's well being. It is evident, therefore, that it is 
essential for politicians, institutions and service providers to be well acquainted 
with the opportunities that the digital world can be a source of, in order to 


ultimately strengthen services and inter-personal relationships. 


What is left to deal with is the difficult task of fighting common misbelieves 
about the digital world. This is yet another example of the “fear of the new", or 
what Freud called a “conservation struggle", linked with the idea of a “death wish" 
— in other words, the instinct that pushes us to protect whatever we have at the 
price of sacrificing our future and hopes. How can we then reassure those who fear 
that digital networks are replacing real life relationships? How can one argue that 
new technologies are opportunities instead of burdens? How can skeptics be 
convinced that digital connections get us closer together, instead of tearing us 


apart? Generally speaking, how is it possible to humanize technology? 
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It is perhaps possible, through a global collective shared discussion, to outline 
that the digital world is a crucial element to sustain the needs of our communities 
allowing us to rethink education and social services, in terms of more inclusive and 
flexible policies. Overall it appears that in order to translate the digital 
opportunities available into successful solutions for our communities, it is 


necessary to look at the issue in a different way. 


If we don't help citizens envision the possibilities that their realities offer; 1f 
we don't create a new system of values people can look up to; if we don't promote 
change; then, going beyond the “fear of the unknown", or the “Pillars of Hercules", 


will never be possible, and we will never cross the new horizons that lie ahead. 


Prof. Eugenia Romanelli, italian journalist. 


romanelli@bazarweb.info 
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INTRODUCTION 


High Dynamic Range (HDR) images represent larger dynamic range and wider color 
gamut compared to Standard Dynamic Range (SDR) images. To reproduce HDR images on 
displays with limited dynamic range, a process known as tone mapping is required. Tone 
Mapping Operators (TMO) aim at mapping the larger dynamic range to smaller one by 
preserving some of tonal levels and dismissing the rest, such that the visual quality of tone- 
mapped image is comparable with that of the original one. Several TMOs rely on histogram 
equalization [1]. Indeed, histogram equalization efficiently redistributes tonal levels to 
preserve information where the pixels' density is the highest in the intensity domain. 


PERCEPTION-BASED TONE MAPPING 


Tone mapping using histogram equalization associates a luminance range in the SDR 
domain (assuming the general specifications of SDR displays) to each luminance range (bin) 
in the HDR domain. However, such an approach cannot differentiate relevant information 
from noise in a uniform background or in dark areas. To address this issue, our proposed 
scheme first compares the number of distinguishable tonal levels in the HDR luminance range 
with those in the SDR luminance range. This is achieved using the Perceptual Quantizer (PQ) 
[2], which is derived based on the peak values of Contrast Sensitivity Functions (CSF). The 
CSF defines the human vision contrast detection threshold for different spatial frequencies 
and background luminance. Wherever the number of tonal levels is higher in the SDR range 
compared to the HDR one, we modify the mapping curve to reduce them. Our technique then 
redistributes those tonal levels to increase details in highlights and/or to achieve SDR images 
with higher overall contrast. 

Our experimental results show that such an approach can greatly improve the tone 
mapping of night scenes as well as scenes with uniform background such as a large sky area. 
Additionally, artistic effects such as fading are also preserved using our proposed scheme. 


CONCLUSION 


We proposed a method to address the main issue of histogram equalization-based tone 
mapping operators, which is their inability to differentiate visual information from noise. Our 
technique relies on a model derived from psychophysical experiments to prevent non-visible 
information in HDR images to become visible in the tone mapped ones. Our method is 
especially efficient for dark scenes and preserving the artistic effects such as fading. 


References 


[1] G. W. Larson, H. Rushmeier, and C. Piatko, “A visibility matching tone reproduction 
operator for high dynamic range scenes," JEEE Transactions on Visualization and 
Computer Graphics, vol. 3, no. 4, pp. 291-306, 1997. 

[2] S. Miller, M. Nezamabadi, and S. Daly, “Perceptual Signal Coding for More Efficient 
Usage of Bit Codes,” SMPTE Motion Imaging J., vol. 122, no. 4, pp. 52-59, May 2013. 


49 


FROM ENCYCLOPÉDIE PLATES TO VISUAL KNOWLEDGE FOR 
CRAFTSMEN: THE PROJECT VISUOPLANCHES 


Marcantonio Catelani!, Maria Teresa Zanola?, Clara Vecchio? 


‘Università degli Studi di Firenze, 
"Università Cattolica del Sacro Cuore, Milano 


marcantonio.catelani@unifi. it 
mariateresa.zanola@unifi. it 
clara.vecchio@uniparthenope. it 


Encyclopédie plates have shown the importance of visualisation: Diderot and d’Alembert 
commissioned several hundred engraved images depicting artistic crafts and common trades in 
preindustrial France, specifically aimed at illustrating their entries. Since pictorial representations of 
workmen and shops are not so frequent, the descriptions and explanatory plates in the Encyclopédie 
remain highly important both to scholars of decorative arts and manufacture and to researchers of 
social and economic history, so as to the current understanding of historical techniques and 
technologies. 

The plates provide a space of non-verbal representation which is profoundly effective for 
communication among producers, craftsmen and the general public: they describe the concept and fix 
the relationship between the idea, the corresponding picture and the final term. 

Many of the images are not technical drawings but genre scenes portraying a small group of workers 
in their activities. The plates from the Encyclopédie often represent stages in complex technical 
processes by juxtaposing images of different types. Vignettes representing human figures engaged in 
various activities are supported by large-scale renderings depicting tools and their proper manipulation. 
Encyclopédie plates have a typical iconic structure: the work is anticipated by the title, which is divided 
into two distinguished sections. At the top - almost on the third of the plate — the place of production is 
represented, where men and women act inside the workshop; in the lower part, all the tools presented in 
the workshop are illustrated, are seen in details, and numbered with the corresponding term. The 
actions are analysed one after the other, as if a zoom gradually passes through the sequences, focusing 
on the most important details. 

The project Visuoplanches aims to rebuild the history of the relationship between the image and the 

classification of terminology to transfer arts and crafts knowledge, starting from the models established 
during XVII" and XVIII" centuries in Europe. Visualising the tools, in combination with legendas, 
establishes a reference model for tool classifications and for their effective description: the text 
accompanying the engravings introduced to the descriptive method which was proposed to beginners 
destined to become experts in the future. Visual representation confirms the encoding effort and 
dissemination of new findings. From plates to modern digital image, tools and technical items continue 
their dialogue between users and craftsmen, between producers and sellers. 
Visual representations show arts and crafts which will transfer technical and specialised knowledge: the 
analysis of some cases will assess the dynamics arising from the dialogue between definition, 
narratives and visualisation, which reveals to be a sustainable method for transferring technical 
knowledge and presenting terminology effectively. Various forms of painting and engraving — from 
artistic drawings to computer-generated images — show craftsmen's knowledge and their need to spread 
it out. 
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ABSTRACT- Being Namibia, it is common that both urbanization, the use and influence 
of Over-The-Top (OTT) services and applications has changed society’s behavior 
towards culture. Many have little cultural knowledge or neither to pass on to their 
young ones. Advantages of urbanization have influenced a tradition of domestic 
travelling amongst the youth; however know little of natural surroundings and the 
underlying intriguing history. In the paper, we would like to discuss and demonstrate 
our mobile-web application “rock art, fauna flora and landscape game” which we 
developed to influence and preserve indigenous knowledge and traditions. We will also 
discuss the server, client and crowd sourcing components of the application. 


KEYWORDS: Flora, Rock art, mobile-web application, serious games, indigenous 
knowledge. 


1. INTRODUCTION 


African Music, Arts, Literature and Cultural practiced, have always played a 
significant role in the daily lives and practices of Africans. Even so, it has gained 
more interested and respect from all over the world, as the world becomes more 
familiar with the rich tradition of the continent [1]. The arrival of western philosophy 
of development, haltered, suppressed, outlawed, neglected the indigenous knowledge 
and traditions of Africans [2]. It is unfortunate that the African norms are no longer 
frequently practices and applied as common as they were before. This could be 
influenced by the colonial past experienced amongst developing African countries, 
which has changed a country's Governance, its people and country. The colonial 
struggle had many go into exile where they furthered both their studies at tertiary 
level and joined the armed forces. In finding its own identity or rebuilding its identity, 
skills and workforce is often needed in achieving the goal. This process attracted an 
increase in urbanization to the cities; which were once originally created for its 
natural resources, as most jobs were created in these cities [3]. 


“Namibia has a rich bio-diversity, which has remained untapped. Coupled with 
extensive indigenous knowledge’ [2] relating to the use of flora, fauna products etc. 
The push versus pull factors of urbanization play a critical role in the destruction of 
[3,9] indigenous fauna and flora in the parts of the country, as land is increased for the 
demand of housing, or for infrastructure that would damage the land, to provide for 
the majority population in the city. It is thus very important in preserving the 
Indigenous Knowledge and Traditions of the country and in parallel influence the 
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behavior of the youth, in understanding and appreciating the cultural and historical 
value. ‘Knowledge still remains amongst the elderly, hence the need to collect, 
analyze, preserve and appreciated" [2 ]. It is by these studies, we designed the 
application faflora "Rock art, Fauna, Flora and landscape’, to bridge the gap and 
orientate society's behavior towards indigenous knowledge. 


1.2 Concept 


Based on the results from a survey we conducted amongst a small group of 
Namibians, we observed that the process of urbanization plays a role on an individual 
one especially in adapting to the new environment. One often finds they having to 
apply both their cultural norms and those developed to best shape and fit their current 
environment as well the community in which they find themselves. A trend that has 
had an impact on the youth of the country as they battle with *intercultural identity, an 
outcome of adaptation. This poses a risk, whereby they are neither part of, nor a part 
of the host culture. As a result finding it difficult to acquire much of the cultural 
knowledge, practices, and other indigenous factors’ [5], see.figl. It is due to this fact 
that the youth have very little knowledge of their cultural norms and practices, as a 
result of early urbanization see.fig2. On a case study conducted by Suzanne LaFont a 
staggering 80% of those from a rural background revealed the need to preserve 
cultural history, whereas those from the city felt it as somewhat important" [6], see 
fig.3. This shows the role of one's geographical location on the cultural knowledge. 
Urbanization advantages such as employment opportunities have seen travelling 
trends amongst the youth increase, a luxury for those who can afford it. Information 
remains unknown due to limited knowledge about the particular surroundings or the 
rich cultural heritage as depicted in rock-art found in popular tourist destinations. 


Count of How well do know culture? 


Count of where were you born & raised 


not that 
well, i 
lived in 
the city, | 
forgot 
much 


intermi... 
45% 


Fig.1: Count on the Fig.2: Influence of the 
cultural knowledge birthplace on culture 
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Taste 13: How important is it to preserve Namibian traditions? 


OYO cohort 80% 
PS cohort 32% 58% 10% 0% 


Fig.3: Importance of 
preserving Indigenous 


2. SYSTEM ARCHITECTURE 


The application comprises of a game and crowdsourcing section for uploads of 
indigenous content to be added to the application. For validation purposes, a user is 
obligated to create an account, for which they can upload and view content. The user 
can either upload a quiz or application content (fauna, flora, rock-art & landscape), of 
which all are used in the ‘faflora’ (flora, fauna, rock art and landscape) application, 
and added to the gallery. Binary Large Objects (BLOB) such as images is stored in a 
file system for performance reasons, whereby a request is made by the client to the 
server to retrieve. If a view or quiz is requested, the server returns the .hbs files from 
the handlebars template system, to client. 


3. IMPLEMENTATION 


Controller Gets Data from DB, & 
Pushed to hbs Template 


Fig.4: Schematic System Architecture 
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Client and User Interface: Express handlebars.js, HTML5+CSS 
Handlebars are a JavaScript compiler and an extension of the mustache JavaScript 
template language, known for its ‘logic-less’ template engine, which keep the pages 
clean and decoupled from logic-based files; this makes the app much more responsive 
and interactive. CSS styling and HTML is added for the User Interface. 


Database: MySQL was chosen for due to its strong consistency and robust 
database design. It will store user profile and application content. Binary Large 
Objects (BLOB) such as images is stored in a file system for performance reasons. 


Server: Node.js using Express.js & Handlebars 


Express is a minimal and flexible Node.js web application, with its core features in 
handling the client communication. This code in fig. 5 handles request to randomly 
load different quizzes and draw quiz images from the database. 


function loadRandomQuiz() { 
return query('SELECT * FROM quiz’) 
.then(({ rows }) => ( 
let quiz = randomElem(rows) 
let img_ids = [quiz.hint_image_id, quiz.correct_image_id, 


quiz.answer1 image id, quiz.answer1_image_id, 
quiz.answer2 image id, quiz.answer3 image id, 
quiz.answer4 image id, quiz.answer5 image id] 


Fig.5: Server code to load random quiz 


3.1 Solutions and Demonstration 
Upon loading of the application, the user is directed to the home page where they can 
select from either 4 elements, Rock art, Fauna, Flora or Landscape. Once either is 
selected, they can either choose to view the gallery. Amongst other tabs, the user can 
click on the play button, where they are redirected to the game page. Each Quiz has a 
title, question and related question and hint images, which are retrieved from the 
database. Once the question is read the user can navigate between hint images, for a 
possible answer of which each has a description to it. A confirmation is given as to 
whether or not a selected answer is correct, where they can use navigation tabs to 
proceed to the next or previous quiz. Should a user want to contribute a quiz or related 
content, registration as a contributor is required, where they contribute indigenous 
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findings to the application. The contributed content with their details are displayed in 
the gallery, and or the quizzes upon validation, see fig.6 and 7. 


Rockart 


View Gallery 


Fig.6: Application homepage 


To which ethnic 


B a & s i 3 


Fig.7: Application game page 


4. SUMMARY 
Based on our results compared to those of Suzanne LaFont results, a much greater 
percentage of the participants who live in the city are unaware of the cultural and 
traditional indigenous knowledge. This could be due to a number of factors previously 
discussed in the paper such as the role of Urbanization. Those who live in the city are 
also much more reluctant to preserving Namibian traditions as opposed to those from 
the rural regions of the country. In conclusion we can deduce that, more of the 
Namibian youth are likely to make use of the application, as nearly half rarely travel. 
This unaware of the Rock art or Indigenous fauna or flora found in parts of Namibia, 
as a mere majority prefer to travel either alone or in a group of friends when 
travelling. This could be seen as ‘a potential for domestic or regional tourism growth, 
in expanding the countries tourism sector substantially'. With the use of a mobile 
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phone, this would make it much easier for the user to become increasingly aware and 
curious as to the surrounding parts of the country. Enticing them to travel and explore 
the different areas. The use of the application could also be used as a reference tool 
for appreciating the cultural heritage and pre historical use of the applications 
elements, Fauna, Flora and Rock art. Which is a step towards changing society's 
behavior in preserving the land and communities in which these indigenous elements 
are found. 


5. DRAWBACKS AND FURTHER WORK 
The current application demonstrates the contribution of indigenous knowledge from 
an ICT literate person and the Namibian indigenous and traditional elements. The 
contributors range from academics, professionals and contributors with access to ICT 
services. However, almost two-thirds of Africa's population [7], and the majority of 
the elderly in Namibia still live in rural areas. This shows that a rich traditional 
knowledge can still be found in rural areas. “The democratizing, empowering promise 
of a mere presence of new media technology is far overstated, as many people are still 
without access to the web, [7] hence the need to draw from, and make information 
accessible to low literacy users in underdeveloped regions of Namibia and that of 
Africa. The next step of the application will see how we can acquire knowledge from 
the elderly in underdeveloped regions of Namibia with hopes to extend that to other 
African countries, for games centered on broader African traditions, landscapes and 
practices. ‘Knowledge of the users’ culture, values, natural experiences within the 
environment and perception improve users interaction with ICT services [7]. Thus 
designing an application, taking into account, a graphical user interface with 
interaction elements that depict the socio-cultural environment of the targeted users, 
which has proven to yield a significant success [5,6]. Alternating in a crowd-sourcing 
application for the elderly in rural areas, as contributors to the application and quizzes 
on different African countries! 
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ABSTRACT 


In the last years, in the manufacturing sector, there has been an increasing diffusion of 
digital technologies, so that the term "Digital Manufacturing" emerged. These technologies 
have entailed an impact at the corporate level, through the necessary reengineering process, 
and socially, making the customer the protagonist in the production of their purchases and 
enhancing their creativity. This change occurred in response to the reduction of the life cycle 
of products and to the need to meet the needs of its customers in the best way.# 


3D printing is an innovation which, together with digital technologies, has enabled the 
diffusion of digital manufacturing, facilitating the transition from the paradigm of mass 
standardization to that of mass customization. The expiration of some patents related to print 
has encouraged the diffusion of additive manufacturing; now it is increasingly used by 
consumers and enterprises. Since its birth, it has been applied for prototyping, while in 
recent years also for the production of products and spare parts. The sectors where it is 
most widespread are the automotive, the aerospace, the medical and the artistic with 
relevant feedback in the Cultural Heritage field. 


3D printing is considered a disruptive innovation because, encouraging the sale of files 
instead of a sale of items, it can optimally solve economic and social problems that have 
been emerging in recent years.Ht is considered a technology with strong growth, but it may 
encounter obstacles in its expansion due to the problems concerning the protection of 
intellectual property: file sharing can encourage the spread of non checked 3D files and then 
it leads to the free duplication of digital objects and the consequent violation of intellectual 
property. A natural consequence may be the failure to control the authenticity of the files, 
and then the risk is reflected in the spread of counterfeit and uncontrolled goods that 
endanger public health.#The lack of specific legislation to protect file properties further 
aggravates the threat of the illegal use of the same.#Currently the use of 3D watermarking 
and encryption of data is a solution to this problem, although an action at national and 
European level which protects the dissemination of unauthorized files is still desirable. 


For the future, if the issue on intellectual protection is effectively resolved, a further increase 
in the deployment of such printers is expected, especially if it is accompanied by a decrease 
in the price of 3D printers, which is expected as a result of patent expiries and the 
development of technological progress 
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Abstract — FTV (Free-viewpoint Television) is an innovative visual media that allows users to 
view 3D scenes by freely changing their viewpoints. MPEG has been developing FTV standards 
since 2001. The first phase of FTV was MVC (Multi-view Video Coding) and the second phase of 
FTV was 3DV (3D Video) for multiview display application. The current third phase of FTV 
targets super multiview, free navigation and 360 3D applications. In this paper, new 3D scene 
representation is proposed for free navigation application of FTV. 


INTRODUCTION 


FTV (Free-viewpoint Television) [1]-[7] is an innovative visual media that allows users to 
view 3D scenes by freely changing their viewpoints. MPEG has been developing FTV 
standards since 2001 [8]. The first phase of FTV was MVC (Multi-view Video Coding) [9] 
and the second phase of FTV was 3DV (3D Video) for multiview display application [10]. 
The current third phase of FTV [11] started in August 2013. This MPEG-FTV standardization 
targets super multiview (SMV), free navigation (FN) and 360 3D applications [12]. 3D scene 
representation is one of the key technologies of FTV. Multi-view plus depth (MVD) 
representation was developed at the second phase of FTV. When the MVD is applied to the 
FN application with large baseline of the third phase of FTV, degradation is observed at the 
object boundary of the synthesized views. It is caused by inaccurate depth data of the MVD. 
In this paper, new 3D scene representation is proposed to suppress such degradation. 


PRINCIPLE OF FTV 


FTV was developed based on ray-space representation [13]-[16]. In the ray-space 
representation, one ray in the 3D real space is represented by one point in the ray-space. The 
ray space is a virtual space. However, it is directly connected to the real space. The ray space 
is generated easily by collecting multi-view images while giving consideration to the camera 
parameters. 

Let (x,y,z) be three space coordinates, and (0,9), where -m/2x0€m/2, -n/2<p<n/2, be the 
parameters of direction as shown in Fig. 1 (left-top) for the orthogonal ray-space. These ray 
parameters construct a 5D ray-space (x,y,z, 0,9) of multiview images. We define a function 
fy, z, 0,9) whose value corresponds to an intensity of a ray. This ray-parameter space is the 
"ray-space." It is clear that ray-space is 6D if time is included as a parameter. 

Although the 5D ray-space mentioned above includes all information viewed from any 
viewpoint, it is highly redundant due to the straight traveling paths of the rays. Thus, when we 
treat rays that arrive at a reference plane, we can reduce the dimension of the parameter space 
to 4D. Although the 4D ray-space gives both horizontal parallax and vertical parallax, the 
horizontal parallax is more important than the vertical parallax. If the vertical parallax is 
neglected, ray-space becomes 3D with parameters (x,y,0). 

Thought the parameterization is simple in orthogonal ray-space, this parameterization 
cannot represent all rays; for example, rays parallel to plane z=0 (9—n/2). Thus, spherical ray- 
space shall be defined as shown in Fig. 1 (left-bottom). Given a reference plane with the 
normal vector in the direction of (0,9) , where -m<0<7, -n/2<p<7/2, and coordinate of (6), 
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the ray emitting from the surface of an object perpendicular to the reference plane at 
intersection of (€7) is defined by fé7,0,¢). 


Definition of Orthogonal Ray-space Example of orthogonal ray-space and rendering for 
à p" linear camera arrangement (p=0) Section 1 
i p d Intensity: f 
Intersection 


Direction: (6, p) Section 2 


u=tan@ Section2 


S(x,y, u=tan 6) 


Example of spherical ray-space and rendering for 
circular camera arrangement (9=0) 


Section 1 
Intensity: f 
Direction: (8, p) 


LENA 


Fig. 1 Ray-space representation and view synthesis [4]. 


The following explains how ray-space is captured, constructed, and free-viewpoint images 
are synthesized. For g=0, the orthogonal and the spherical ray-spaces can be captured by 
using linear and circular camera arrangements, respectively. Ray-space is constructed by 
transforming captured images by cameras in real world into ray-space domain, in which 
transformed images are aligned in parallel. The image planes in orthogonal ray-space are flat 
planes whereas sine shaped plane in spherical ray-space. Fig. 1 (middle) depicts examples of 
the constructed ray-space for orthogonal ray-space and spherical ray-space when g=0. 
Constructed ray-space can be used for free-viewpoint image generation. Given the location of 
virtual viewpoint, a free-viewpoint image can be generated by cutting the ray-space vertically. 
As illustrated in Fig. 1 (right), in orthogonal ray-space the vertical cross-section is a flat plane 
whereas in spherical ray-space is a sine shaped plane. If the captured images are not dense 
enough for free-viewpoint image generation, ray-space interpolation should be performed to 
obtain a large number of views. In this process, virtual views are placed between real camera 
views as if the camera arrangement is very dense. 


CONVENTIONAL 3D SCENE REPRESENTATIONS 


Figure 2 shows major 3D scene representations in the view-geometry domain. In this 
figure, horizontal axis denotes the number of views “N”, where views are images of a 3D 
scene captured by multi-camera system. Vertical axis denotes the number of depths “M”, 
where depth means depth map. A depth map is an image, in which the intensity of each pixel 
represents distance between the optical center of a camera and the surface of an object in the 
3D scene. Depth corresponds to the geometrical information. Many depths can be represented 
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in a different form, e.g. a 3D model. A camera position has view and/or depth. It means 
number of views and depths can be different in a multi-camera setting. When M-N, all 
camera positions have view and depth. However, if N>M, there are some camera positions 
that have only view, but no depth. On the other hand, for M>N, there are some camera 
positions that have only depth, but no view. A general form of view-geometry representation 
is multiview plus multi-depth (N View + M Depth) that covers all other presentations. Details 
about different representations will be given as follows. 


3D model 


ANS 5 
Texture Multiview + Multi-depth 


(N View+M Depth) 
Point Clouds 


Surface 
Lightfield 


One View 
+ Multi- 
depth 


Number of Depths M 
(Geometrical Information) 


One view + 
One depth 


Multiview Ray-space 


Single view 


Number of Views N 


Fig. 2 Conventional 3D scene representations in the view-geometry domain [4]. 


Representations with no geometrical information are as follows. The conventional 2D-TV 
system is achieved by a single view image. Representation with more view than one view is 
multiview representation. If number of views is very large, multiview representation is 
equivalent to ray-space, which is explained in the previous chapter. 

Representations in the following are for the case of having only one view with variable 
geometrical information. Using one view plus one depth representation we can achieve a free- 
viewpoint image generation in a narrow range. By adding more geometrical information or 
having multi-depth, we can represent the free-viewpoint system as one view plus multi-depth, 
such as LDI (Layered-Depth Images) [17]. LDI contains more geometrical information, so 
that it can provide wider range of free-viewpoint. With all geometrical information or 3D 
model of the scene, we have 3D model plus texture representation that can provide a wide 
range of free-viewpoint. Point clouds can be converted to 3D models [18]. 

Surface lightfield [19] contains less amount of geometrical information in comparison 
with 3D model while several views are required for this representation. This representation 
can provide wide range of free-viewpoint images. (N View + N Depth) is MVD (Multi-view 
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plus depth) representation adopted at the second phase of FTV by MPEG. GVD (Global View 
and Depth) [20] that consists of base view, base depth, residual views and residual depths is a 
compact data format of MVD. Redundancy of MVD is removed in GVD. 

In [21], various 3D scene representations are surveyed in one dimension, in which the 
more geometrical information corresponds to the less number of view and vice versa. As 
shown in Fig. 1, it is along the axis connecting the most left-top point, i.e. 3D model/texture, 
to the most right-bottom point, 1.e. ray-space. 


NEW 3D SCENE REPRESENTATION 


Concept of the proposed method is shown in Fig. 3. Most of the rays of a 3D scene are 
captured as rays crossing a reference plane 1. However, there are other reference planes for 
objects that can't be represented well by the reference plane 1. It is denoted by a reference 
plane 2 in Fig. 3. If the reference plane 2 is set to be the surface of the object, the reference 
plane 2 collects rays emitted from the object. However, it can be set to enclose the object as 
shown in Fig. 3. In this case, the reference plane 2 collects not only rays emitted from the 
object but also rays from background near the boundary. 


Reference Plane 2 


Reference 


C Plane 1 


Positions 


Fig. 3 Concept of proposed method. 


Figure 4 shows data of the proposed method. The left of Fig. 4 denotes data collected by 
the reference plane 1. For example, view | is a view image at camera position | in Fig.3. 
There are gray portions in the view images. The gray portions correspond to rays from the 
reference plane 2 and those data are not necessarily needed because they are collected by the 
reference plane 2. 

The right of Fig. 4 denotes data collected by the reference plane 2. There are 2 
representations for them. One is view images on reference plane 2 and another is orthographic 
projection images. The latter is the same as the spherical ray-space representation. However, 
its range is limited to the rays from the reference plane 2. 

Views are synthesized as follows. First, rays are reproduced by using data on the reference 
plane 1. Then, rays reproduced by using data on the reference plane 2 are added. 
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Data on Reference Plane 1 Data on Reference Plane 2 


View 1 View images on reference planc 
or 


Orthographic projection images 
View 2 


View 3 


View 4 


View 5 


---- 


Fig. 4 Data of the proposed method. 


CONCLUSION 


New 3D scene representation is proposed for free navigation application of FTV with large 
baseline. It is effective to suppress degradation observed at object boundaries of synthesized 
views. 
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Abstract — The THz-ARTE project proposed to combine together visible (Vis) and near 
infrared (NIR) hyper-spectral imaging technique with data acquired in the Gigahertz (GHz) and 
Terahertz (THz) frequencies for the study and analysis of 2-D polychrome artworks. 


INTRODUCTION 


The non-invasive diagnostic and analytical approach to the study of works of art is an 
important topic within the priority research area of “Technologies applied to Cultural 
Heritage” [1]. Several techniques have been developed in the past to achieve a 
characterization of paintings as complete as possible and to extract data on the provenience 
and age of artists’ materials [2]. Among the imaging techniques a very important role is 
played by Infrared Reflectography (IRR) and Vis and NIR hyper-spectral imaging (HSI) 
techniques [3-7]. The former enables the visualization of details underneath the visible surface 
of a painting, exploiting the partial transparency of the painted layer when subjected to IR 
radiation. HSI, in addition to the IRR information, provides a spectral and colorimetric 
characterization of the entire painted surface. 

Within the “Terahertz Advanced Research Techniques for Non-Invasive Analysis in Art 
Conservation” (THz-ARTE) project the Italian (ENEA-Frascati & CNR-IFAC, Firenze) and 
the Japanese (NICT-Tokyo & NNRICP-Nara) teams proposed to extend the imaging 
techniques mentioned above to Gigahertz (GHz) and Terahertz (THz) frequencies, more 
specifically in the range from 20 GHz to 0.6 THz, which is situated between the microwave 
and infrared regions of the electromagnetic spectrum. 

Radiation in the THz and GHz spectral range is reflected by metals and aqueous materials, 
while it is transmitted with low attenuation through most dielectric materials, such as paper, 
several pigments and dyes, binding media [8]. Regarding the characterization of materials 
used in works of art, GHz and THz imaging systems will provide useful data on revealing the 
internal physical structure of non-metallic objects without touching the artworks investigated. 
This non-invasive cross-section image of the object is acquired by extracting the reflected 
pulse from a particular interface of two media with different refractive indexes [9-15]. This 
type of information will complemented the data obtained with Vis and NIR HIS thus 
producing a more complete set of records to be used by other colleagues, conservators, and 
curators. 

In the present communication, some of the first results obtained within the THz-ARTE 
project as well as the application of the IFAC-CNR Vis-NIR HIS (Fig. 1) and the ENEA 
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TGHz/Hz 3D (Fig. 2) systems on the fresco-painting on “tavella” depicting St. John Baptist 
(approx. 1700) by Alessandro Gherardini will be reported. 


Fig. 1 Vis and NIR hyper-spectral imaging scanner developed at IFAC-CNR 


Fig. 2 GHz/THz 3D scanner developed at ENEA 
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Abstract 


The passage of time scans some historic steps that, to this day, can represent real secrets that lie behind 
the image of a work of art. 

Our usual way of observing, studying and examining the Cultural Heritage, does not always provides an 
adequate tool to understand the hidden "truth" and, for this reason, the secrets that can be revealed make the 
world of art one of the sectors with unrivaled charm. 

It is to the figure of the restorer who, not infrequently, the fate entrusts the most extraordinary 
discoveries: behind the overlaying of dirty patina, yellowed varnish and old repainting, you can uncover the 
truth camouflaged for centuries behind the hidden veil of time. 


INTRODUCTION 


Although, since the beginning of the eighteenth century, the figure of the restorer already 
represented an autonomous profession, history witnesses how it was customary to privilege his own 
creative flair compared to the one of the artist whose work he was going to preserve. 

The restorer of the time felt entitled to "reprocess" the work of art by relying on his own 
interpretation, rather than conserve and preserve with rigor the integrity of the original artifact. 

So many works of art, over time, were even "modified"; to the point that, today, it is legitimate to 
speak of actual "tampering". 

In the nineteenth century, thanks to the Italian and French experience, the difference between 
"mechanical restoration", strictly limited to the restoration of the painted surface, and the "artistic 
restoration": a real scientific methodology dedicated to the study of techniques for cleaning and 
reintegration of artifacts, started defining. 

This awareness leads us to scan the necessity to adjust and subordinate the conservation 
interventions to the "constitutive materiality" of the artifacts, revealing the fundamental need to 
know, examine and evaluate in advance all those restoration processes that, over time, could 
undergo inevitable changes that would eventually cause what we can define as extensive damage to 
the Cultural Heritage. 

The need to establish an "ethical code" designed to regulate the professional training of the 
restorer, definitely matured in 1931 when the ICOM (International Conference of Museums), 
reaffirmed the importance of this issue by signing a document that saw historians of art and 
museum directors, which confirmed the need to define the training of the professional figure of the 
restorer. 

Over the last decades, the skills of some great experts, as Cesare Brandi (Director of the Central 
Institute for Restoration in Rome) and other important realities, have provided a valuable 
contribution to define the role of the person to whom was entrusted the task of preserving an 
artwork; by raising their awareness to subordinate their work to very specific rules which would 
establish techniques, technologies and methods: according to a logic oriented to the strict protection 
of the original material. 
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Investigating a work of art 


The preservation of Cultural Heritage, its conservation, enhancement and continuous 
rediscovery, provide a precious contribution and the essential foundation for the affirmation, 
development and diffusion ofthe identity of all people. 

This awareness can and should lead the professional figures of this sector to never limit 
themselves to the mere appearance of what they are admiring, but to aspire to investigate the search 
for a truth that may have been concealed over time. 

While examining a work of art we need to linger to also observe the most negligible details and 
ask ourselves questions that, although apparently obvious, they can reveal the suitable path for the 
encoding of a lost language and / or forgotten through the centuries. 

The fact that among the best known prerogatives of authoritative teachers of every era there is to 
conceal, behind the image of a work of art, hidden meanings not easily understood, which often 
remained secret for centuries is not negligible. 

Think of the Sistine Chapel. 

The most majestic Chapel of the world represents just one of the best known symbolic art icons 
and most studied in the world; but it is also one of the most irrefutable testimonies of the ability ofa 
great artist, who was Michelangelo, to express ideas and thoughts steeped with transgression; 
enunciated through symbols skilfully camouflaged in an absolute masterpiece, that even the client 
(exactly the latter) would have been able to code. 

What is most fascinating is that the deep expertise of these artists made them so erudite to 
succeed in wisely eluding as well as offhandedly the strict rules imposed by the client. To the point 
that, five hundred years from their representation, to this day, they still provide the cue for 
publications and revelations made by leading experts ... dedicated to unravel mysteries still 
disguised today behind the veil of that time which is waiting patiently to preselect the person to 
whom fate will entrust the privilege to perform the most daring revelations! 

This "refine" as well as complex and acute ability to express thoughts and concepts that only the 
most acquainted could understand, went far beyond the doctrinal canons established by the usual 
iconographic "codes" and privileged the artist with an absolute freedom of expression, forged in the 
knowledge and also the courage to even challenge the papal commission revealing, without its 
knowledge, the most transgressive concepts and allegorical meanings. 

But the passage of time has seen even less competent figures alternating, even though unaware, 
who have concealed and seriously penalized the preservation of the asset, both in its physical 
consistency and in the preservation of its historical and iconographic meanings. 

Not infrequently the interventions of clumsy "restorers" have ended up in tampering the original 
appearance of many works of art and changing, inevitably, also the iconographic code represented 
in them. 

This dynamic has been, and is still today, a harbinger of unconfessed secrets hidden behind 
patinas of dirt and yellowed varnish; this makes the work of art a treasure chest of mysteries, since 
hundreds of years, waiting to be revealed. 

During the restoration of a painting that seems apparently intact, the removal of the layer of dirt 
has made resurface the original composition, just as the artist had designed and manufactured five 
centuries ago. 

Is it possible to sense the presence of secrets that lead to the research of truth to be revealed? 

Can the Virgin Mary, be portrayed without the veil? 


Extravagant tampering and the research for the truth revealed. 


The iconographic tradition that, through the ages, has seen the figure of the "Virgin and Child", 
represented is very complex and varied and develops iconographic types that will undergo from the 
simplest to the most significant mutations. 
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Often these different types of depictions which represent a fundamental precondition for the 
rediscovery of the artifact; Since they allow to locate fundamental historical passages to identify the 
various customs and artistic traditions to which they relate: both from a doctrinal aspect, rather than 
in purely artistic and creative terms. 

The Western Cultural Heritage, in fact, is often characterized by its complex and mutable 
iconographic typology: understood not only as an interpretation of a variety of themes and 
narratives; as well as mutations, often also substantial, characterized by multiform as discontinuous 
historical and iconographic representation of the same subject. 

Although these mutations, occurring in time and in space (namely in different periods and 
geographical areas), objectively coexist in a single reality, they themselves determine an eclectic 
and very complex scenery that, not infrequently, precisely interposes the correct interpretation the 
work of art. 

The usual iconographic tradition sees the Virgin Mary portrayed with a red vest and wearing a 
blue cloak that leads back to the colour of the sky and that associates the image to the "Regna 
Coeli". 

For centuries the Virgin has been represented with the crown that, until the thirteenth century, 
was laying over a coif usually coloured of red and beneath it the hair was collected: the 
"maphorium". 

It is necessary to wait for the representation of the Stoclet Madonna, an opera of Duccio from 
Buoninsegna (Siena, 1255-1318 / 19) in Brussels today, in order to see that the coif is removed 
from the tradition and replaced with a white veil interposed between the hair and a blue veil. 


t pu EN 
Duccio from Buoninsegna ca. 1290-1300 

There are many reflections that can be detected in the complex and significant development of 
the iconographic tradition related to the representation of the "Madonna and Child": as the 
interesting comparisons and references regarding the pagan representation of the Egyptian goddess 
Iside while accommodating her son Horus on her laps. 

The testimonies that we receive from the Marian texts coming from Egypt, in fact, represent a 
source of knowledge that significantly supplies the "recovery" of the iconographic path that allows 
us to understand the origins of the Marian representation from the III-IV century and on. 

Consider that the depictions of the Virgin dating from the early centuries will lead, later, also to 
the doctrinal connotations placed in connection with the heresies. 
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With the Council of Ephesus, in 431, the figure of the "Madonna and Child" was placed 
permanently in connection with the image of "human motherhood", marking the beginning of an 
iconographic tradition that remained, in centuries to come, extremely flourishing. 

The icon of "Platytera", the solemn Byzantine Madonna often seated on a throne with the Child 
on her knees, depicts the two figures leaving broad interpretation to the tenderness of the maternal 
relationship. Mother and son are facing each other while Mary, with her right hand, gently pats the 
shoulder of the Child; they mark an iconographic tradition that remains constant for centuries. The 
figure of Jesus, although over centuries of representations, interprets the regal and solemn aspect of 
the Savior: symbol of divine salvation. 

The "monumentality" of the figures represented in these early centuries, will characterize a 
Virgin absorbed in meditation and will mark a long period in which Jesus will be tacitly put beside 
the Virgin; but always in a loving relationship punctuated by detachment and where the glances 
between Mother and Son never will meet. 

The imperturbability "of this representation eloquently communicates the doctrinal content of the 
iconographic message, which interprets the Virgin Mary as the one who bears witness to the 
mystery of the Incarnation of God's Word and His intercession for mankind; but, simultaneously, it 
focuses and also directs attention to the figure of the Child. 

Only at the beginning of the eighth and ninth centuries, the iconographic typology of the "Virgin 
of Tenderness" will appear for the first time, which will reunite the two figures in a more 
affectionate and involving attitude, where the faces will approach until they touch and the attitudes 
will mark the apex of loving mutual encounter. 

The representation of the sentimental emotionality between Mother and Son will provide 
inspiration, manifested in the artist as well as in the client, to subordinate the rigid canons of 
classical iconography (up to this moment so well marked by the councils as well as the 
iconographic tradition), to a more passionate and confident scheme, but also more refined and a 
harbinger of greater creativity. 

"Virgin and Child" will indulge in those tender and accomplices gestures dictated by the love 
that unites Mother and Son; in which mankind naturally recognizes itself, creating an interpretative 
freedom, that from the monumental rigor of the early centuries, gives way to a representation 
permeated with the tenderness of maternal feeling, which will become the actual protagonist of the 
composition. 


Leonardo da Vinci 1478-1482 Hermitage Museum Saint Petersburg 
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Mysteries hidden behind old restorations and a revealed truth 


Before restoration (fig. 1 After restoration (Fig. 2 
g g 


Can the Virgin Mary, be portrayed without the veil? 

The experience of the restorer teaches that, over the centuries, a large amount of works of art, 
even though with the intent to be preserved, have ended up with the most varied types of 
interventions of "restoration." 

Clumsy artists, declined into the role of improvised conservatives of products made by their 
predecessors, have often interpreted the Cultural Heritage as an asset to intervene on with no will 
and, what is worse, feeling also entitled to "improve" through renovations, additions and personal 
interpretations that have tampered and compromised the integrity and authenticity of the products. 

For a combination of various reasons and circumstances, a good restorer must be as aware as 
possible of what he going to preserve. 

An example for everyone is the choice of the solvent, which is one of the most delicate decisions 
because its task is to remove the harmful layers. However, a wrong choice, is likely to irreparably 
compromise the work of art. 

Each step must represent the result of a careful prior evaluation, to be weighted with diligence 
and to be carried out after an accurate preliminary investigation, which has allowed to investigate in 
depth the characteristics of the original artifact and subsequent contributions. 

It will be appropriate to define the nature of the original materials, their characteristics, types of 
harmful layers to remove and many other factors. 


Observing at the work of art depicted in image 1 (Before restoration) has given the possibility to 
establish that it is a painting done in tempera on wood, dated from the XVI century. 
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The transparency of the colours, due primarily to the fact that the pigments have been mixed 
with a water-binding agent, allows to appreciate a clear and accurate preparatory drawing, by means 
of which the artist has outlined the entire compositional setting. 

The spread of the colours is characterized by a palette with gaudy and striking chromatic colour 
range, made using pure colours; kneaded with each other on a basis of a criteria of "essentiality." 

A considerable layer of dirt and a yellowed varnish camouflaged a previous restoration, dating 
from several years ago. 

But my attention has been "captured" by a striking particular: the incongruity of the 
representation of the image of the Virgin from the usual iconographic norm. 

The flowing hair of the Madonna didn't show any trace of the veil! 

Although it is true that the iconographic typology of the Virgin over the centuries underwent 
various iconographic interpretations, over the centuries, it is also known that the attribute of the 
veil always remains an indiscriminate constant. 

The absolute incongruity of this element has induced me to also observe the fact that the drafting 
of the pigments with which the hairstyles were made, including that of the Child, St. Joseph and St. 
John the Baptist, had a paste and a "fullness" that, noting it better, was very different from the 
chemical characteristics and the material consistency of the rest of the painting. 

Impatient to unravel the mystery concealed behind the image of this representation of the "Virgin 
and Child", and after having previously consolidated the limited areas at risk of "loss of colour", I 
started the intervention of removal of the harmful layers. 

Although I had already sensed that the painting had been subjected to substantial "repainting", 
their removal has proved to be extremely exciting! 

The cleaning, little by little, has made the delicate veil resurface, almost transparent and tinted of 
a light blue pastel pigment that perfectly harmonized with the colour of the cloth lying on the lap of 
the Virgin Mary and on which the Virgin lovingly housed the Child, standing next to Her. 

Traces of pure gold, which originally finely decorated both the veil of the Madonna and the 
crowns of the Saints, have re-emerged under the coarse repainting. 

Once the cleaning has been finished and the compositional integrity of the work has been 
restored, it has been necessary to intervene with the pictorial reintegration of damaged parts, which 
made us re-establish the gaps, paying particular attention not to "overflow" to the original colour. 

The completion of the restoration has allowed to appreciate again the iconographic coherence of 
the composition, recovering the original iconographic canons. 

They testify obvious references to the passage between the most ancient iconographic doctrine, 
where the Virgin is intent to caress the Child with her right hand, and the traditional iconography of 
the "Virgin of Tenderness", that combines the figures of the icon in the attitude of a loving mother. 

The painting has been subjected to the study by Professor Mina Gregori who returned the 
authorship to the tuscan artist Francesco Brina (Florence 1540-1586). 

Vasari reports his activity, also witnessing that, in 1565, he worked on the construction of the 
apparatus for the wedding of Francesco de 'Medici and Giovanna of Austria. 

The works of Francesco highlight a rather archaic culture, with frequent references to Andrea del 
Sarto; but also influenced in many ways by the complex structure of the Florentine altarpieces of 
the time, made by contemporary Giorgio Vasari. 

Francesco, although a student of Michele di Ridolfo along with his brother Giovanni, is not to be 
confused with the latter because of a pictorial quality of different level. In fact, his style is 
characterized by a much more refined quality; ; so that many works of art are still of controversial 
attribution today between the hand of Francesco and that of his master Ridolfo. 
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CULTURAL ASSETS, “EXTENDED” MUSEUMS AND THE 
HISTORIC CITY: THE CASE 
STUDY OF AN APP FOR GENOA 


Valentina Fiore, Lauro Magnani, Sara Rulli 
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Università degli Studi di Genova 
Genoa, Italia 
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Abstract - Getting tourists in normally unaccessible spaces, showing sounds and events to 
the citizens, all this while causing the same emotions the assignor had felt, increasing the value 
of spaces and corners of the city that still haven't been properly inserted in a touristic route, 
reaching a high divulgation to a large public: these are the goals set to the research project for 
an App aiming to spread information about the city. 

The App's software and user interface, developed by Xenia Production and Smartland 
Project s.r.l., are made to contain different levels of information, from the whole urban area 
to the single artistic artifact, inviting the visitor to experience a real route in the city and a 
virtual one in the in-depth analysis of its cultural aspects. 

The App is focused on the Strada Balbi and Piazza dell’ Annunziata area, where a big part 
of the University's buildings are located: these buildings - which together form a public 
*extended museum" lived, inhabited and studied directly by the University - allow the 
academic environment, its researches, and the new discoveries made by the continuous studies 
on the artifacts it lives in to speak for themselves. In this regard the App, working with the 
large patrimony property of the Athenaeum, is meant not only for the external public, but 
also for the internal one, meaning the students themselves, for which it will become a learning 
tool and an instrument for a more direct experience. The results of the researches for the 
study projects, final papers, in-depths analyses and topicalizations will be immediately and 
directly communicated to the tourists and, why not, the city's inhabitants. 


INTRODUCTION 


The use of multimedia technology, and particularly of virtual reconstruction, allows art 
historians to not only study the dynamics of monuments, examining in depth different, well defined 
points of view and to contextualize them in their original environment, but also to share the results 
of the research with as many people as possible, providing categories and complex interpretations 
for artistic artifacts, and more importantly, for the city in its whole. Multimedia technology can thus 
help researches, becoming a testing ground for hypotheses, a way to verify interpretations and a 
valuable medium for the divulgation and representation of researches and the achieved results all at 
the same time. Throughout the years studies have been conducted in these fields by the 
D.LR.A.A.S. department in collaboration with research groups afferent to the genoese Polytechnic 
School and young Ligurian companies (see: Acknowledgements), creating a relationship based on 
interchange and collaboration that aims to entwine liberal arts and scientific and technological 
know-how. The focus has been put on promoting the genoese Atheneum's assets and the city in 
general, which is full of palaces (declared UNESCO World Heritage in 2006), frescoes and art 
collections to be studied in their complexity and with a systematic overall view. Getting tourists in 
normally unaccessible spaces, showing the citizens sounds and events, all this while causing the 
same emotions the patron had felt, increasing the value of spaces and corners of the city that still 
haven't been properly inserted in a touristic route, reaching a high level of divulgation to a large 


76 


public: these are the goals set to the research project for an App aiming to spread information about 
the city. The App's software and user interface are made to be as flexible as possible, containing 
different levels of information, from the whole urban area to the single artistic artifact, inviting the 
visitor to experience a real route in the city and a virtual one in the in-depth analysis of its cultural 
aspects. 


Designing the App: the graphic interface and communication with the user 


The needed requirements for an easy, immediate use and the aspects entailing the interface with 
which the App presents itself to the user, menus, accessibility and choice of itinerary, maps, content 
visualization formats, social network accessibility and online experience sharing have all been fine 
tuned. 

The graphic interface (see: Acknowledgements) is divided into three main parts: the left side 
menu (for the main actions), the right side menu (for the contents), and a central part (for 
visualizing the contents themselves). 

This way the user can hold its tablet or smartphone horizontally like a steering wheel, without 
needing to turn it each time; the positioning of the commands on side menus allows the use of the 
two hands without moving them on the screen. 

The choice of white characters on a black background helps defining a clear interface, with a 
high contrast ratio even under intense lights. 

The left menu, the main one, contains all the login and social network connection commands (the 
App is constantly connected to the internet and to social networks), but also those for sharing with 
other users the experiences made in the city ( "Share" button) and information on the position of the 
user (“Where am I?" button). 

The “Itineraries” button grants access to a screen that contains a matrix made of more buttons 
whose number is equal to the routes the user can choose to get to know the city. After choosing one 
(eg. the Aquarium-Nunziata-Balbi-Aquarium ring) a preview screen appears showing the itinerary 
and the position of the user on an aerial map. 

The *map" button shows the position of the user, a step-by-step view of the route and the 
succession of the various points of interest, each independently clickable. 

The right menu grants access to each point of interest's in-depth information (by selecting them 
independently) and allows the user to zoom the map in and out, change the point of view, eg. from 
above (satellite view) or a three quarter view from the four cardinal points (bird's-eye view, using 
the button on the bottom right). 

A following implementation phase of the App will allow the user to mark points of interest other 
than the default ones, using a remote or local database with remote synchronization and the 
possibility to edit the map with filters, specific visualizations, etc. 

The central area, which can be viewed in full-screen, is used to visualize videos, pictures and 
text. 


Itineraries of valorization of the historic city and the university 
center: the Aquarium-Nunziata-Balbi-Aquarium ring 


The App invites the user to physically - or whenever this isn't possible, virtually - enter and 
discover significant and peculiar points of view that, even in usually open buildings, aren't easily 
accessible, but can be proof of the assignor's particular approach, the layering of the urban 
structure, views, and relationships between interiors and exteriors, such as the various nymphaea, 
atriums and hallways behind the front doors, often corners of gardens, openings in the high density 
medieval urban structure. This App is also made to overcome the existing architectural barriers: the 
audio, video and textual supports can also help people with hearing, visual or physical disabilities to 
thoroughly enjoy these otherwise unaccessible artifacts. 
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To obtain an effective design of the App there was the need to analyze the historical sources and 
the images of the city in order to consider the “city artifact” as a complex and multi-layered 
structure where, in the different phases of the its development, historical events and topographical 
realities have been shaped by the actions of the Republic's government, the aristocratic families and 
the religious orders, following the needs of the economic, productive and port activities. 

To draw the guidelines for the App's development (for the realization of the software, see: 
Acknowledgements) two different methods of approaching the urban artifact were adopted: on one 
hand to discover the city following topographical routes in which fundamental points of interest are 
marked, on the other hand to a thematic in depth analysis of some aspects or events of the city 
regarding specific artists that have worked there (eg: Galeazzo Alessi) or cultural and artistic 
monuments that strongly characterized the city's appearance, a UNESCO World Heritage since the 
organization recognized the peculiar social and economic identity of its manneristic and baroque 
palaces, full of internal decorations. 

The results of the feasibility studies regarding the App and its resulting realization allowed the 
conception of an innovative way of divulging historical and artistic information to a public that has 
no specific academic education. The App's innovative character, with complex facts explained in an 
easy and quick way, makes the product absolutely competitive with the potential to become, with 
further development, a real business for the involved companies. 

The repetitive format chosen for presenting the artifacts, divided in three categories for 
explaining each cultural asset (“Architecture” - “Decoration” - “Thematic Itineraries”), further 
divided into sub-layers, together with the easily usable interface of the App, which allows to easily 
access and choose the itineraries, the maps, and to share one's experiences online thanks to the 
possibility to connect to social networks, makes the product a real and concrete model that can be 
exported to other territorial realities other than Genoa, both in Italy and abroad. 

The App focuses on the area of Strada Balbi and Piazza dell'Annunziata, containing descriptive 
and explicative text regarding the Chiesa della Santissima Annunziata, Palazzo Chiavari Belimbau, 
Palazzo Balbi Cattaneo, the Ex Collegio dei Gesuiti and Palazzo Balbi Senarega. Inside them are 
sited many different and extraordinary realities of the genoese Athenaeum. These buildings - which 
together form a public “extended museum" lived, inhabited and studied directly by the University - 
allow the academic environment, its researches, and the new discoveries made by the continuous 
studies on the artifacts it lives in to speak for themselves. In this regard the App, working with the 
large patrimony property of the Athenaeum, is meant not only for the external public, but also for 
the internal one, meaning the students themselves, for which it will become a learning tool and an 
instrument for a more direct experience. The results of the researches for the study projects, final 
papers, in-depths analyses and topicalizations will be immediately and directly communicated to the 
tourists and, why not, the city's inhabitants. 

The contents are structured following four different types of text: a general, introductive one 
briefly but efficiently explaining the historical artifact as a whole, a descriptive one that analyzes 
the architectural aspects of the asset, an explicative one about the decorations, and a specific one 
illustrating some peculiar data about the church or palace. 


The three dimensional virtual reconstruction as a way to immediately 
comprehend the city's and the artifacts? historical phases 


The possibility of offering the users virtual, tridimensional models of the different phases of the 
historical, residential and urban realities of the city during their visits is equally fruitful. This allows 
to show them in real time the beauty of the architects’, artists’ and the assignors’ works. 

As a consequence of choosing Piazza della Nunziata and Via Balbi as the focal point of the 
itinerary, the attention focused mainly on the base-line study of the different building phases of 
Piazza dell’ Annunziata, an area that has undergone lots of transformations, continuously altering its 
appearance between the XV and the XIX centuries, and of its buildings, particularly Palazzo De 
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Ferrari Chiavari Belimbau. With the goal of analyzing the different historical phases of the square 
and the changes brought to the buildings needing renovation, some of the most important 
iconographic and cartographic sources have been selected and verified to be utilized as guidelines 
for explaining the user an historically complex urban situation. 

A good example would be Palazzo Balbi Senarega, which allowed providing the user with the 
three dimensional virtual reconstruction of the architectural situation at the beginning of the XVII 
century, then modified and expanded, since the middle of the same century, with the addition of the 
garden and the nymphaeum that characterize the palace. The initial phase of the project - testified 
by Rubens’ drawings from J Palazzi Moderni di Genova, published, in a second edition, in Antwerp 
in 1652 - is thus presented to the visitor and compared with the new spatial values of the mid- 
century renovations, allowing him to immediately understand the importance and dimension they 
have meant for the building and its relationship with the medieval city. The architectural extensions 
and the new decorations have shifted the residence’s style to purely baroque. In this case, the 
potential of three dimensional reconstruction will allow the visitor to experience the genoese 
architectural dynamics and the different choices of the assignors. In this phase the focus was 
particularly on the choice of the layout for the final rendering of the reconstructions: avoiding the 
use of a “mimetic” approach that can be confused with reality, the choice fell on a more “abstract” 
version excluding explanations regarding the use of materials and that can easily appear as a virtual 
"maquette", a study model of a reality that doesn’t exist anymore. 


Further development: the Renaissance city of Galeazzo Alessi 


For the fifth centennial of the birth of architect Galeazzo Alessi and in preparation for an 
international conference on this outstanding architect who had worked in Genoa in the second half 
of the XVI century some thematic routes — which will be developed with the collaboration of Dep. 
DSA of the genoese Polytechnic School — have been hypothesized to connect and bring visibility to 
his works, not only for tourists, but first and foremost for the Genoese. This kind of itinerary can 
serve an autonomous or functional and of in depth analysis purpose to the topographic ones. It can 
be a starting point to develop a method for museums (eg. the possibility to develop an App for 
museum itineraries) and contemporary galleries. 
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FIGURES AND TABLES 


Fig. 1: The “Aquarium-Nunziata-Balbi-Aquarium Ring” 
Fig. 2: A view from the top of the medieval tower of Porta dei Vacca, one of the "Points of Interest" 
of the “Aquarium-Nunziata-Balbi-Aquarium Ring" 
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PERCORSO 1:ACQUARIO-NUNZIATA-BALBI-ACQUARIO 
3H 30' 2Km 


Il percorso ti condurrà nell'esplorazione non solo degli edifici e delle percorrenze più note ma, 
soprattutto, di quegli aspetti e scorci poco conosciuti nei quali più spiccatamente potrai cogliere il 
carattere della città e dei suoi edifici. Potrai vedere superfici murarie "sfondarsi" in illusionistici 
affreschi, chiese medievali trasformate secondo i canoni del barocco, grandi dimore nobiliari sorte, 
in epoca rinascimentale, su preesistenze tardomedievali e oggi godibili nella rinnovata 


magnificenza di opere ad affresco e stucco, atri dilatati in inaspettati spazi di giardino e 
scenografici ninfei. 
XK PUNTI DI INTERESSE 


Fig. 3: Presentation of the “Aquarium-Nunziata-Balbi- Aquarium Ring”: the “Points of Interest” 
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PALAZZO BALBI SENAREGA 


Costruito per i fratelli Giacomo e Pantaleo Balbi a partire dal 1616 dall'architetto Bartolomeo 
Bianco, la dimora, oggi sede della Scuola di Scienze Umanistiche, presenta l'inusuale soluzione di 
due identici piani nobili sovrapposti. Potrai entrare nell'atrio del palazzo, spingerti fino al ninfeo, 
visitare le sale interne e apprezzare così il gusto già pienamente barocco con cui Francesco Maria, 
figlio di Giacomo, rinnovò la dimora. Dalla galleria del piano nobile, a fianco di una ricca collezione 
antiquariale di busti, prende avvio, con la "favola" del Ratto di Proserpina affrescata da Valerio 
Castello, un grandioso ciclo decorativo di celebrazione della committenza che coinvolge, nello 
spazio del mito, tutte le sale dell'appartamento e vede al lavoro i maggiori esponenti del Seicento 
genovese: Domenico Piola, Gregorio De Ferrari e Giovanni Andrea Carlone. 


Fig. 4: Palazzo Balbi Senarega: 3D reconstruction of the architectural situation at the beginning of 
the XVII century 


Fig. 5-6: Palazzo De Ferrari Chiavari Balimbau: virtual tour 
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Abstract — 3D has been widespread for several years now, but for various reasons has not yet found 
its "everyday use" in the field of cultural heritage. There are, of course, applications specifically 
designed for museums, but they usually lack an advanced educational use. Most of the examples 
consists of a mere catalogue of manipulable objects, but the 3D format has the potential to create 
much more complex learning activities, which at the moment are unfortunately not yet widespread 
and certainly not standardized. Museo Galileo is testing a series of possibilities to develop new 3D 
services, keeping in mind that, for educational and dissemination activities, they have to be deeply 
integrated with the more classic ones. 


1 STATE OF THE ART OF 3D USE IN CULTURAL HERITAGE 


In the last twenty years the tools for the production of 3D models have evolved to a point that 
place them outside the pioneering phase and introduce a new stage of daily use. 

The most popular software houses offer online services both for the creation of models from 
photos and for printing and handling shared models. In addition, 3D printers have reached 
affordable prices, although their spread is still limited. Interesting examples of 3D usage in the 
cultural heritage domain are shown in a list compiled by Archeolab [01]. Among these are the 3D 
Virtual Museum [02] with a collection of geotagged artifacts that can be manipulated [03] and 
consulted by clicking on numbers that return textual information superimposed on the model; the 
Google Art project [04] that allows you to combine text, images and video to 3D handleable 
elements; the Smithsonian [05] and the British Museum. This technology has also been pursued by 
several European projects, such as 3D-COFORM [06], which contributed to the 3D model metadata 
adopted by Europeana. The examples described in the above list are certainly valid, but when we 
consider the user interface and interaction, the current models do not go much beyond the simple 
manipulation of objects, thus not benefiting from the full potential of the 3D interface. The situation 
is certainly destined to evolve in the near future, thanks to a more technically advanced offer and 
the achievement of a usage standard recognized by all users. 


2 HTMLS AND THE NEW TECHNOLOGICAL OPPORTUNITIES FOR 3D 
IN WEB AND MOBILE ENVIRONMENT 


The efficiency of the fruition method for 3D models is obviously the crucial point for the spread 
of their use and has always been the weakest link in the whole chain. To achieve this there are two 
possible solutions. The first relies on specialized hardware, such as stereoscopic visual simulators 
and systems for augmented reality, in which the content provider has the whole production chain 
under his control, while the second expects from the producer only the 3D model that will be shown 
on the user's system. While the first is an experience reserved for the few, the second mode could 
potentially involve all users of the web, so it is worth investigating this aspect further. In this case 
both the hardware and the software are beyond the author's control and the risk is that the user 
experience is not the one imagined or, worse, the 3D model cannot be displayed at all. Obviously 
for a successful visualization of the model, an agreement on standard and its spread among a critical 
mass of users become mandatory. In the past, after a period of proliferation of ad hoc systems, two 
formats have spread, VRML and Flash, the latter of which was later banned for safety and 
efficiency reasons. 

In the meantime the WebGL and HTML; libraries were adopted by the most popular browsers. 
WebGL library provides all the basic functions for 3D on the web: other high-level graphic 
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libraries such as X3DOM, Skecthfab, and recently Unity, are based on it. Since Skecthfab is a 
proprietary software and Unity is not yet mature in its web version, the most viable solution is the 
X3DOM standard [07]. X3DOM is the integration in browsers of X3D [08], which is, in turn, the 
evolution of VRML. The X3DOM framework is maintained by a nonprofit consortium and is a 
member of the W3C. At present this standard is implemented in almost all popular browsers and 
can be used on both PC and mobile environment, so it has all the characteristics to be successful in 
the long term and to allow an even wider uptake of 3D models. To allow an easier spreading of 3D 
even among implementers not too accustomed to this technology, additional software-layers, such 
as 3DHOP [09], were built in order to make available a standardized interface and mode of 
interaction, with some limitations, but without the need to resort to a massive use of programming. 

Museo Galileo opted for the use of native X3DOM for maximum freedom in the implementation 
of the various features and the widest possible circulation. 

Itis also worth mentioning that the metadata standardization adopted by Europeana, is beneficial 
for a greater distribution of 3D models. 


3 THE DIDACTIC-COMMUNICATIVE PERSPECTIVE AND THE 
INTEGRATION WITH EXISTING PRODUCTS 


Sharing its own collections is, for a museum, a mission, as well as a possibility to increase the 
number of visitors and to engage a younger audience. Beyond the technical issues, the biggest 
deficiency in this field is the poor number of these models. The limitations are mainly of two kinds: 
on the one hand there are few models available since, in addition to the technical difficulties of 
acquiring certain materials, the creation of a realistic 3D model requires scanner acquisitions or 
photographic campaigns that need substantial resources; on the other hand the possible means of 
interaction are usually quite limited, even in relation to the fact that the available examples are often 
pertinent to archaeological or historical artifacts only. There is therefore considerable room for the 
development of user interfaces that allow much more advanced uses, especially in the presence of 
special collections such as those preserved in the Museo Galileo. 

The Museum has been working on 3D models since the early 90s, but initially the usable 
products have been limited to the reconstruction of the physical models and the realization of 
movies that showed their operating principles [10]. Over the next decade a few interactive models 
were developed in Flash [11]: these provided the user a more active approach to information. The 
current need to implement systems that do not use Flash and also reach out to mobile platforms, 
must be seen as an opportunity to rethink from scratch the possible ways of user interaction with 3D 
models and to start a debate on the possible uses of 3D for historical and scientific museums. 

The new technological opportunities are the necessary basis for the development of new 
applications, but for a successful result they must be part of an educational and communicative 
process that allows the user an integrated fruition with the services currently offered by the 
Museum. The road ahead therefore contemplates both the integration of the technology into the 
educational activities already proposed to individuals and schools by the Museum staff, and the 
evolution of the cultural offer in the guise of a support for the onsite visit and the autonomous 
learning, activities that have been pursued for many years gradually updating the technology, 
focusing first on the Web and then also on mobile platforms. From this perspective the 3D format 
can be considered the highest point in a series of integrated tools such as the Multimedia Catalogue 
of the Museum [12] and the Scientific Itineraries in Tuscany [13] and the upcoming videoguide 
App for the onsite visit. A close examination carried out inside the Museo Galileo, from a didactic 
point of view, has proved that 3D can be used in various ways inside the Museum, also considering 
that, due to the complexity and variety of its collections, it covers a large series of situations that 
can usually be found in different kind of institutions. The result is a series of possibilities on which 
we started to work as a basis for a development of new services. 


83 


e The manipulation of the object works well only with simple items, consisting of a single 
element without moving parts, such as the statues. These kind of objects are represented, even if to 
a small extent, within Museo Galileo. The benefits of this mode are a chance to look at the object 
from all points of view and the possibility to see or study it remotely. The production of these 
models in some cases presents objective difficulties related to the accessibility of all parts of the 
object and to the difficulties of displacement in the case of heavy objects. 

e The downloading and printing of the models are instead well suited both for remote usage 
by students, and as stand-alone continuation (for individuals and groups) of the experiences made in 
the educational workshops at the Museum. The models can comprise a single piece or several 
movable parts. Depending on the case the model can span from a mere artistic reproduction (e.g. a 
statue) to a faithful reconstruction of interactive items used to simulate the operation of a real-world 
object. In the latter case, the utilization of the model requires not only the ability to download the 
3D file, but also the operating instructions that must be provided contextually. For practical reasons 
it is not possible to offer the printing service inside the Museum. 

e Some of the most significant potentials of the 3D can be fully appreciated while developing 
the interaction with the model. In the case of simple tools, interactivity may be achieved by 
highlighting several significant parts showing predefined views with self-explanatory contextual 
pop-ups that include various media. Most of the instruments of the Museum are composed of many 
moving parts and pieces that cannot be touched by the public. It is therefore essential that the 3D 
objects also allow the user to interact with the relative movements of their parts. Interactivity can be 
developed at various levels: from the simple split of the model which outlines and describes its 
various components to the simulation of the movement of these parts, or even further to a learning 
sequence that, with the help of a list of instructions, guides the user toward a result aimed to teach 
the use of the instrument. 

e Inthe case of incomplete, destroyed or never actually built instruments, the 3D is the best 
way to show them in their original form in order to facilitate their understanding by the visitors. 
However, for the end user the system cannot be limited to the presentation of objects, but must also 
give the necessary information to activate and use them and to understand their operation principles. 
This technology can allow curators to verify the feasibility and functionality of tools that are known 
only through their description. 

e An understanding of some instruments, even complex ones, is greatly facilitated if these are 
shown in motion and eventually put in their original context. The possibility to interactively choose 
different points of view makes it an even more effective system than a simple video. 

e Finally, a further use of 3D is the exploration of environments. The ability to recreate virtual 
environments offers many opportunities that will become even more significant as the quality of the 
representation will increase. This mode is particularly fruitful in the case of areas that cannot be 
opened to the public or no longer exist, and in the reconstruction of virtual environments in which 
to place interactive objects that can be connected to multimedia descriptions. 

All these possibilities are the basis of 3D application to museum services: however it is 
important to keep in mind that, for educational and dissemination activities, 3D must not remain 
just a showcase designed to exhibit the ability to manage an innovative technology, but becomes a 
new service integrated with the more classic ones and an additional way to achieve the 
communication objectives of the Museum. These technologies are used to build an interactive 
environment that enables easy and pleasant user experience in order to stimulates the learning of 
subjects traditionally considered boring and difficult while maintaining the scientific accuracy. 


4 EXAMPLES 


The main possibilities that are being considered and will be presented both in the individual 
applications and in a dedicated section are: 
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Object manipulation 


Fig. 1 - Bust of Galileo Fig. 2 — Waldseemuller's globe 


The bust of Galileo [Fig. 1], exhibited in the room dedicated to him, here is not contextualized 
and is explorable from all angles with a simple drag and drop. The 3D model was made from 
photographic image processing with freeware software. The service is primarily addressed to the 
remote user who has the possibility to explore the statue from various points of view. 


Download and print of the 3D model 


The Museum!'s bookshop already sells kits of cardboard or wooden instruments to be assembled, 
such as the Mechanical paradox. A printable 3D model could be offered to download. At the same 
time the Educational Laboratories already teach how to perform experiments with instruments that 
can be self-built in the premises of the Museum or at school. In some cases they may also use 3D 
printed parts. 


Interactivity 


In the case of this globe [Fig. 2], in addition to its manipulation, there are predefined viewpoints 
and hot spots that allow to delve into important topics, thanks to pop-ups integrated into the page 
using HTMLS. 


Fig. 3 — Astrolabe 
When we manage complex instruments such as the astrolabe [Fig. 3], it is possible to highlight 


the individual components, providing descriptions for each of them, and to show their relative 
movements. 
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Incomplete or never built instruments 


In the case of 
Leonardo's world 
map [Fig. 4], that 
was transmitted to 
us only ina 
manuscript copied 
by a pupil which is 
hardly readable, 
we reconstructed 
the globe as a 
browsable 3D 
[Fig. 5] model to 
make it 
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Fig. 4-5— Leonardo's globe 
easily accessible to the public, while checking at the same time the consistency of the sketch. 


Animations 


Machines and mechanisms, such as those reconstructed from Leonardo's manuscripts, combine 
the reciprocal animation of the parts, eventually operable, with the possibility of manipulation 
already seen in the previous cases. 


Virtual environments exploration 


Fig. 6 — Virtual Museum 


3D makes it possible to reconstruct environments, even not real ones, that may be interesting 
metaphors to access and understand content. In [Fig. 6] the Galilean instruments have been 
organized in different rooms depending on the time and place in which they were developed, thus 
creating a proper virtual museum that can be explored interactively. Each environment tool provides 
in-depth multimedia cards. 
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S CONCLUSIONS 


The launch of services to the public based on 3D entails a series of costs, because it requires the 
presence and integration of different skills related to content, dissemination, programming, web 
sites production, in addition to those relating to the creation and management of 3D models. In the 
case of the Museo Galileo the presence of an internal Multimedia Laboratory makes it easier and 
more natural this fusion of skills, following a long-standing practice. The institutions that deal with 
conservation, study and dissemination of Cultural Heritage have an interest in providing 3D 
services not because they represent a fashionable innovation, but because the integration of all these 
possibilities with existing multimedia technologies, allows to improve the efficiency of the means 
of study and the quality of services to the public. In our personal experience 3D makes it possible 
both to the visitors inside the Museum and the ones connected remotely to benefit from a useful and 
personalized service, more effective than many classic approach at disseminating the history of 
science, the primary mission of the Museo Galileo. 
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Abstract — Neurosciences, and particularly Cognitive Sciences, study creativity for the implications 
in understanding human and machine approach to problem solving. In fact, subjects such as Cognitive 
Psychology, Brain Neurobiology and Artificial Intelligence, investigate creativity mechanisms related 
to the human brain and cognition. We can state that creativity is a part of every human daily activity, 
so that, to understand individuals behaviour and abilities and the mechanisms of mind, we must start 
from the creative process, as, in general, the search of a different (innovative) solution to a problem. 

In this paper, we will treat the cognitive aspect of the creative process, particularly starting from 
cognitive technologies, devices helping researchers to analyse the process by the point of view of the 
artificial and natural intelligences. 


INTRODUCTION: ART, CREATIVITY AND COGNITIVE SCIENCES. 


“We paint with our brain and not with our hands” (“Si dipinge col cervello e non con le mani”), 
Michelangelo Buonarroti. 


The artistic expressions are universally recognized as creative. This is the reason why Cognitive 
Sciences are interested in Art. In fact, they study artificial and natural intelligences, and creativity is 
one of the most investigated field. Among the questions to which Cognitive Sciences try to answer, 
we wish to recall the following: 

- Creativity is innate or can be acquired? 

- Could we strengthen it, stimulate it? 

- Is there any area of the brain where it resides? 

- Is it a prerogative of natural intelligences or also of the artificial systems? 

From a theoretical point of view, creativity is a complex cognitive process resulting from the search 
of a balance between conscious and unconscious processes. Indeed, creativity may be considered a 
borderline state of mind, in which the thought seems to fluctuate in a fluid cognitive state. When a 
new idea arises to the consciousness, and then a balance is achieved, the mind turn back to a 
"creative-off" state and divergent thinking is replaced by canonical thinking. 

Through creative, our brain/mind system is able to find original solutions, new perspectives, further 
ways to organize the incoming data so as to give rise to an enriched reality. The experience we 
have of the world is made up of details and information, but is also rich in complex forms that 
interact each other not only to shape concepts and meanings, but also to evoke emotions, memories, 
thoughts, which are not directly matched with some physical features of a given stimulus detected 
by the senses. Consequently, creativity is a basic cognitive process which mechanisms are wired in 
our brains in order to give to a whole experience of the world, even when we see it for the first time. 
Hence, these processes allow the human brain to enrich our experience, shaping the deeper 
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motivations that guide our cognition well beyond contextual needs. This is the cognitive core of art 
and creativity. 

However, the creative cognition gives not rise to an illusionary world: simply, it extends mind 
possibilities, opening new frontiers and unblocking landscapes and options, thanks to an “as if? 
experience. Furthermore, this view of the brain working through "as if" processes may be integrated 
with the theoretical perspective by Changeux [1]. This author argues that we should always 
consider the perception as a creative process. For instance, when someone observes something (e.g. 
an artwork) in his/her brain a sort of re-creation is “in march", so to find a meaning but also to 
attribute mental states, emotions, intentions to the “source” of that object, thus mentally tracing the 
path from the creator to the observer. 

In Cognitive Sciences, creativity is often studied investigating brain functions such as memory, 
language, attention, neuro-motor abilities, through the cognitive evaluation of the external stimuli 
given to a brain, as in Lindsay e Norman [2]. This cognitive evaluation is, obviously, subjective, 
because it is influenced also by social factors and cultural context of every individual, as suggested 
by Vygotskij [3, 4]. Also Piaget [5, 6], as Vygotskij, underlines as the interaction between subjects 
and objects stimulates the constitution of superior psychic processes. Nevertheless, Vygotskij 
expresses an interesting point of view, strongly related to the consideration of the links between 
creativity and means (also technological ones) at disposition. In fact, he underlines how the human 
ontogenesis is determined also by the contribution of the cultural instruments at disposition in the 
historical and social context. 

The links among Art and Technology, creativity and the study of its mechanisms are several: the 
opportunity for the preservation of the artistic heritage; the potentiality provided by new 
instruments for the artistic expression; the possibility to study the links among Art, brain and 
Technology and, jointly, the creative processes, allowed by the progress of Artificial Intelligence, 
Brain Imaging and technological devices. 

In this introductory paper, we will focus on the last point of the list. In the next paragraph we will 
introduce why and how scholars perform scientific investigations on creativity. In the following, we 
present the use of technology to express creativity and an experimental example of it, concluding 
with our considerations. 


SCIENTIFIC INVESTIGATION ON CREATIVITY 


Scientific investigation through new technological tools, such as Brain Imaging and Artificial 
Intelligence (A.I.), opens a wide scenario on the opportunity in studying relationships among Art, 
Technology and brain, and, consequently, creativity. Brain Imaging techniques allow notable 
improvement in the analysis of the brain "in action", that is in the study of the individuals’ reactions 
and brain mechanisms involved in motor, cognitive or perceptive tasks. Thanks to devices and 
invasive or not methods, such as fMRI (functional Magnetic Resonance), MEG (Magnetic 
Electroencephalography), PET (Positron Emission Tomography), EEG (Electroencephalography), it 
is possible, in fact, to verify in real time the response of an individual to specific stimuli. Because of 
its low invasivity and high time resolution, EEG is the most used technique to investigate 
mechanisms such as creativity. Intuitively and experimentally, it is possible to state that Art is 
conceived by artists and observers thank to the neural activities of the brain, creating the aesthetic 
experience [7]. The study of this mechanism represents the base of the experimental discipline 
named Neuroaesthetic, officially born in 2001, studying the neural bases of the contemplation or of 
the creation of Art. Founder and pioneer of Neuroaesthetic is Semir Zeki, known neurobiologist 
who, in the *90s, begun to study to the links between Art and brain, using psychological tests and 
electroencephalography. Zeki compares the artist to a neuroscientist [8, 9, 10], "exploring 
potentiality and the ability of the brain, even if with different tools" and, even with a different 
language, transmits a knowledge discovered or intuited. Art, so, is not a prerogative of the artist, but 
resides also in who look at a masterpiece. Think to Duchamp, who has ever underlined that the 
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meaning of a work art is given not only by the artist, but also by who look at the work, through 
auxiliary interpretations. The aesthetic experience of an artwork integrates and transforms the 
individual perception of reality in a lived experience with regard to the subject: the artwork disturbs, 
excites, soothes the individual and the like. Finally, Aage Brandt [11] proposes to consider neuro- 
aesthetics as the study of neuronal process of perception and mental organization of cognitive 
activity stimulated by the artwork, following both a cognitive and evolutionary approach. 
Particularly suited for the described research, the B.C.I. (Brain Computer Interface) headsets are 
simplification of the medical equipment for EEG [12], and allow to record cerebral rhythms and the 
direct brain-computer interaction. BCI devices are widely used in research, for the registration 
completely comparable to the medical EEG, but also for their low cost and high portability. They 
present, moreover, the advantage to keep in comfort the individual wearing them as they allow a 
wide movement freedom in the experimental environment. BCI devices collect several brain 
frequencies, grouped in rhythms: the alpha rhythm (7 Hz — 14 Hz), related to relaxation, meditation, 
contemplation; the beta band (14 Hz — 30 Hz), associated with active thinking, attention, problem 
solving; the delta rhythm (3 Hz — 7 Hz), registered in children and associated with continuous 
attention activity, as in Leeb et Al. [13]; the theta rhythm (4 Hz — 7 Hz), generally related to 
emotional engagement; the gamma signal (30 Hz — 80 Hz), usually related to the cognitive 
interpretation of multi-sensory signals. BCIs allow to investigate the mechanisms of creativity both 
from the point of view of the artist, while creating a work, and from the point of view of the public, 
while observing the final result. 

In this framework, BCI devices are particularly useful in research, either to register the response to 
visual and musical stimuli and recognize the emotions valence [14, 15], and to reveal the 
mechanisms of the visual creativity [16]. The objective of the researches, past and in fieri, is to 
evaluate the emotive and the cognitive response to stimuli, with the aim of understanding what are 
the mechanisms triggering creativity or characterizing the creative process (the insight). In some 
experiments the objectives is to evaluate the emotive and cognitive response to visual-perceptive 
stimuli [17, 18] based on the concept of priming [19]. Other studies, investigate the mechanisms of 
response to colours [20, 21], or to stereoscopy and monoscopy [22]. The obtained results show 
interesting correspondences among some cerebral rhythms and the creative activity. 

Currently, the research focuses on the comprehension of the cognitive mechanisms at the basis of 
creativity, of emotional intelligence and expression, but the technological cognitive tools at 
disposition today show, evidently, an enormous potentiality, in giving the possibility to verify, as 
Vygotskij stated, how the human ontogenesis is strongly influenced by cultural (technological, in 
our Era) tools at disposition in the historical and social context. 


TECHNOLOGICAL EXPRESSION OF CREATIVITY. AN EXAMPLE. 


“Art does not consist in representing new things, but in representing them in a new way" ("L'arte 
non consiste nel rappresentare cose nuove, bensi nel rappresentare con novità."), Ugo Foscolo 


Every creative act starts, generally, in finding a new, creative answer to daily problems. This 
process allow to think “out of the box" to find a solution, the so called “problem solving" process, 
strongly related to problem solving. The term problem solving intend, in fact, the activity performed 
by a living organism or by an artificial intelligence device to achieve a status starting from one or 
more given condition. It is, so, the set of processes allowing to analyse, organize and combine 
information to solve given problematic situations. In its work The Act of Creation [23], Arthur 
Koestler explain creativity through the bi-section mechanism, that is “means to join unrelated, often 
conflicting, information in a new way". In fact, if in our daily life we associate elements belonging 
to the same reference system (book-sheet, cooking-food, etc.), in the artistic, humoristic or scientific 
creation we realize a connection among heterogeneous reference systems, usually considered 
incompatible. Problem solving and creativity are, so, correlated cognitive processes and can be 
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analysed through Brain Imaging methods, and it is evident that we can measure, also quantitatively, 
the activation level of a brain, analysing the electric signal produced. 
With the aim to show how creativity is related to brain activation, we wish to introduce “Mind the 


Chair" (or “La sedia del Pensatore"), an interactive installation focused on a real time audio/visual 
representation of the activity of the brain, providing the possibility to control it. The performance 
allows to visualize the level of concentration of an individual, through an audio/visual answer (LED 
lights and sound generation/manipulation). While using a BCI headset, a person is able to 
understand how *control own brain activity and therefore to increment the level of concentration, 
modifying the intensity of the light or filtering the associated sounds. The interactive performance 
aims to give awareness about how our brain works and what *concentration' means, therefore how 
much brain energy is required to perform a specific task (for example, to increase the lighting), and 
how it can be difficult to keep our brain working for a certain amount of time. 


Figure 1: “Mind the chair", Francesco Soave. The performance installation in action, as exhibited in 
the artistic event “ Terni Festival“, 18-27 Sept 2015, Terni, Italy. 


The concept comes from the idea of 'the chair of the thinker' where a person sits on a chair, and its 
own thought starting from the mind, flows through its body, along the chair and then in the 
surrounding space as light. 

During the performance the visitors are not given physical objects to concentrate on. They are only 
asked to 'concentrate' in terms of 'focusing' on something close to their attitudes. The visitor is 
required to not distract too much from feeling his/her brain activity, or the purpose of the 
installation would fail. During the performance, the word ‘concentration’ is, then, used as the 
simplest way to describe the aim of the interactive installation that is testing the power of the brain. 
For instance, many visitors tried to relax doing some yoga exercises, but even if they were 
expecting a deep fall of the lights, this didn't always happen as, in fact, whatever action you perform 
requires brain activity, even the act of relaxing and 'freeing' the mind. 

The ability to control the brain rhythms and the concentration level, depends, of course, by the 
experience of an individual: solving a simple equation can be an easy task for a mathematician, but 
can be a really difficult task for a non mathematical mind. To make the installation to react is, then, 
necessary to find a specific, personal task that makes our brain work, which, of course, can be 
different for different people. Also, the ability to control the brain changes depending on the 
psychophysical status of the person: if the user is really tired, he/she will experience difficulties in 
controlling the performance. Interesting enough, when the brain activity is peaking, individuals 
show more difficulties in loosing focus and slowing down the level than actually increase it. 
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To realize the project a Neurosky Mindwave BCI interface and specific Java code were used to 
detect the brain activity through the analysis of the collected brain waves. In detail, as the 
installation involves concentration, Beta rhythms play a key role in identifying the level of brain 
activity. To make the representation consistent with the real brain activity, the interactive 
installation does not have a 0 level (in terms of "lights off): this is because our brain is working at 
all times and what makes the difference is the amount of work required to perform a task. Then, 
when the level of concentration is close to zero, a random blinking will appear, to represent the 
floor noise which is, in fact, a random signal existing in every kind of impulse and therefore even in 
the electromagnetic brain rhythms. 


CONCLUSIONS 


Neuro and Cognitive Sciences study creativity and artistic inspiration to discover their cognitive 
process, providing, at the same time, thanks to cognitive technology, the means for new artistic 
expressions, allowing researchers to investigate how the brain creates and understand Art. In 
cognitive science, the functioning of the brain and the achievements of Art are considered together 
to explain our aesthetic experience. The application of the Cognitive Science approach to Art, 
entertainment and educational fields also represents a promising approach. In fact, the importance 
of a BCI-based performance like “Mind the chair" is not limited in creativity studies. We argue that 
the use of BCIs and similar tools, such as, for example, eye trackers, should also be useful within a 
pedagogical program in order to contribute improving users' ability to focus attention on abstract 
cognitive tasks and in translating abstract thinking in concrete operations. Many people find great 
difficulties in approaching similar tasks, since their attention is easily grabbed by environmental 
distractions and so they fail in getting important educational achievements. Furthermore, the ability 
to focus and maintaining attention on mental tasks may contribute to the development of 
multitasking abilities, now considered particularly important in different contexts. Normally, 
expressive arts techniques are useful in helping people to increase attention skills, but they are 
limited by individuals" technical competencies (e.g. drawing) or by personality traits. BCI-based 
tools, such as BrainArt [16], or BCI-based performances, may instead be enjoyed by everyone, 
without any prerequisite both in stand-alone and in group settings. Future research will test if BCIs 
are concretely able to increase attention-based skills and creativity, potentially increasing the 
penetration of art within the realm of the neuro-cognitive approach to individual empowerment. 
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Abstract — There is increasing global awareness - led by the United Nations [UN] - of the 
challenges and opportunities in applying Digital Technologies to the needs and talents of 
disabled and other disadvantaged people. This paper concentrates on major relevant UN 
reports and the proposed EU strategy up to 2020. The Italian situation is addressed by means 
of questions for discussion with participants. The paper aims at stimulating a dialogue to be 
continued at other EVA Conferences during 2016. 
INTRODUCTION 
The principal UN documents identified for this initial discussion paper are as follows: 

- United Nations (UN) Universal Declaration of Human Rights, UDHR, 1948 [1]; 

- UN Convention on the Rights of Persons with Disabilities, UN CPRD], 2006 [2]; 


- World Report on Disability, World Health Organization (WHO) & World Bank, 
2011 [3]. 


- UNESCO Global Report: Opening New Avenues for Empowerment. ICTs to Access 
Information and Knowledge for Persons with Disabilities, PWD, 2013 [4]. 


In addition, two important recent EU documents were also selected: 


- European Disability Strategy 2010-2020: A Renewed Commitment to a Barrier-Free 
Europe, 2010 [5]; 


- A corresponding proposal for a Directive/European Law, 2015 [6]. 


A concluding Discussion includes intended next steps and some questions for discussion 
regarding the Italian situation, 
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THE BASIS: THE UNITED NATIONS UNIVERSAL DECLARATION OF HUMAN 
RIGHTS, UDHR (1948) 


The Universal Declaration of Human Rights (UDHR, 1948,) clearly affirms ‘cultural rights’ 
in Article 22: 


Everyone, as a member of society, has the right to social security and is entitled to 
realization, through national effort and international co-operation and in accordance with 
the organization and resources of each State, of the economic, social and cultural rights 
indispensable for his dignity and the free development of his personality. 


Furthermore, Article 27 states: 


(1) Everyone has the right freely to participate in the cultural life of the community, to 
enjoy the arts and to share in scientific advancement and its benefits. 


(2) Everyone has the right to the protection of the moral and material interests resulting 
from any scientific, literary or artistic production of which he is the author. 


This influential document served to inspire the next milestone document. 


UN CONVENTION ON THE RIGHTS OF PERSONS WITH 
DISABILITIES [CPRD] HIGHLIGHTING TECHNOLOGY’S ROLE 

A subsequent major step by the United Nations built on the UDHR and UN Convention on 
the Rights of Persons with Disabilities [UN CPRD] 2006. 


This crucial document heralded, signposted and legitimised a reinvigorated global campaign 
for Disabled People. Article 9 including substantial attention to Technology and, in 
particular, Accessibility. It has the following key aims (author underlining): 


1. To enable persons with disabilities to live independently and participate fully in all aspects 
of life, States Parties shall take appropriate measures to ensure to persons with disabilities 
access, on an equal basis with others, to the physical environment, to transportation, to 
information and communications, including information and communications technologies 
and systems, and to other facilities and services open or provided to the public, both in urban 
and in rural areas. These measures, which shall include the identification and elimination of 
obstacles and barriers to accessibility, shall apply to, inter alia: 


a. Buildings, roads, transportation and other indoor and outdoor facilities, including 
schools, housing, medical facilities and workplaces; 

b. Information, communications and other services, including electronic services and 
emergency services’. 


This point thus clearly asserts the equivalence in importance of ICTS to Physical Facilities 
and sets out the UN injunction to member states to ‘take appropriate measures’ as described 
a few lines later by two specific types addressing the technology focus of this paper: 
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d. 'Promote access for persons with disabilities to new information and communications 
technologies and systems, including the Internet; 

b. Promote the design, development, production and distribution of accessible 
information and communications technologies and systems at an early stage, so that 
these technologies and systems become accessible at minimum cost.’ 


This seminal Convention stimulated states and UN Agencies to pursue this subject.. 
http://www.un.org/disabilities/convention/conventionfull.shtml [2]. 


WORLD REPORT ON DISABILITY (World Health Organization & World Bank) 


This substantial report [325 pages] provides a comprehensive picture of the situation of the 
world’s health at the end of the first decade of this century. Technology has a clear role 
including attention to ‘Increasing the use and affordability of technology’. Recommendations 
are made to reduce the barriers to information and communication cover: ‘Legislation and 
legal action; Standards; Policy and programmes; Procurement; Universal design; Action by 
industry and the Role of nongovernmental organizations’ . 


Example recommendations include: Accessible information and communication: Consider a 
range of bottom-up and top-down legislative and policy mechanisms including: consumer 
protection, non-discrimination legislation covering information and communication 
technologies and direct obligations on those developing ICT systems, products, and services. 
In the public and private sector adopt policies on procurement which take into consideration 
accessibility criteria. Support the development of telephone relay, sign language, and Braille 
services. When designing and distributing ICT equipment and services, developers should 
ensure that people with disabilities gain the same benefits as the wider population. Producers 
and providers should incorporate accessibility features in the products and services they 
design and sell. Support the education and training of persons with disabilities to take 
advantage of ICT — including training to ensure digital literacy and skills. 


http://www.who.int/disabilities/world_report/2011/report.pdf [3] 


UNESCO GLOBAL REPORT: OPENING NEW AVENUES FOR 
EMPOWERMENT. ICTs TO ACCESS INFORMATION AND KNOWLEDGE FOR 
PERSONS WITH DISABILITIES, PWD 2013. 


In 2013 UNESCO published this major report [Principal author, Michael Blakemore] 
covering the entire world but based in particular on commissioned regional studies as 
follows: (1) Africa; (2) Arab Region & North Africa; (3) Asia Pacific; (4) Eastern Europe & 
Central Asia; (5) [South America, Central America, Mexico] Latin America & the Caribbean. 
Each study was carried out with assistance from institutions and coordinating authors, 
generally from the region. This report was aimed at the following reader groups: 


98 


The report is targeted specifically towards: Policy and decision makers; Local, regional and 
national educational institutions; Teachers, education and information professionals; 
Associations and organizations working with persons with disabilities; ICT professionals and 
industry; and Others interested in ICTs and social inclusion of persons with disabilities. 


Irina Bokova, Director-General of UNESCO, wrote in the Preface: 


*Over one billion people — approximately 15 percent of the world's population — live with 
some form of disability. Facing a wide range of barriers, including access to information, 
education, health care and a lack of job opportunities, persons living with disabilities 
struggle every day to be integrated into society’ 


However, a note of caution was added in the report regarding difficult issues of definition, 
measurement and statistical problems: 


The definition of disability is complex and evolving ---- the recognition of a disability 
varies across countries. Interventions require a sensitive understanding of the disabilities and 
a systemic approach to providing ICTs in the education context. 


However, there can be no doubt that Disability is a major global issue. 


EUROPEAN UNION POLICY INITIATIVES 

Early concerted [pan] European activities in the field began in 1991 with the commencement 
of the EC's first TIDE Programme [Pilot Action: TIDE 0 - Technology initiative for Disabled 
and Elderly people] followed by its successors, aiming to build on and integrate previous and 
ongoing national and regional initiatives. These efforts were given extra emphasis in 2003 - 
the European Year of The Disabled - to raise awareness and stimulate further attention 
including contributing to international efforts. 


In 2010 the European Commission produced an ambitious strategy paper in European 
Disability Strategy 2010-2020: A Renewed Commitment to a Barrier-Free Europe 2010 
[5] . The short (11 pages] forceful document begins with the following clear statement in the 
Introduction. 


'One in six people in the European Union (EU) has a disability that ranges from mild 
to severe making around 80 million who are often prevented from taking part fully in 
society and the economy because of environmental and attitudinal barriers. For 
people with disabilities the rate of poverty is 70% higher than the average partly due 
to limited access to employment. 


This population number of 80 million means that the European disabled — corresponded 
numerically in 2010 to a country the size of Germany (although caution with regard to the 
statistics as indicated above) . Moreover, it continues: 


Over a third of people aged over 75 have disabilities that restrict them to some extent, 


and over 20% are considerably restricted 3. Furthermore, these numbers are set to 
rise as the EU's population ages.’ 
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The European Commission identified eight main areas for action: the first of these was 
Accessibility defined as follows: 


"Accessibility' is defined as meaning that people with disabilities have access, on an equal basis 
with others, to the physical environment, transportation, information and communications 
technologies and systems (ICT), and other facilities and services. There are still major barriers 
in all of these areas. For example, on average in the EU-27, only 596 of public websites comply 
fully with web accessibility standards, though more are partially accessible. Many television 
broadcasters still provide few subtitled and audio-described programmes’. 


After this promising step, the prospect was raised of a corresponding proposal for a 
Directive/European Law to be prepared in 2012. This proposal was released in December 2015. 
http://ec.europa.eu/social/main.jsp?langId-en&catId-89 &newsld=2400. 

The proposal sets out the draft legal document for approval. One striking figure is that by 2020 
the number of disabled people in the EU is projected to be 120 million people with a significant 
market value for special goods and services but fragmented between the EU's 28 countries due 
in considerable part to differing national and regional laws and practices Thus, 'another half of 
Germany s population’ is expected to be added in the current decade. EU supported R & D 
programmes, such as the Seventh Framework Programme (FP 7) and currently HORIZON 
2020, have and continue to make substantial relevant contributions as do also more general 
initiatives and policies at the European, national, regional, urban and rural levels. 


CONCLUSIONS, LIMITATIONS, QUESTIONS & NEXT STEPS 


Principal conclusions include the following: 


- Global and European needs for improved assistance for the Disabled are high and increasing 
as their population ages, but global awareness and momentum appear to be increasing. 

- The UN reports provide a basis for consideration by and for the Arts & Heritage sector 
regarding innovative computer technologies for the Disabled. 

- Cultural institutions such as museums, in cooperation with other actors, can make greater 
efforts for the Disabled using Information Technology as the latter races ahead. 


There are evidently significant gaps in this initial discussion paper, for example International 
Standards and the corresponding bodies such as ISO/IEC and the Web Accessibility Initiative 
(WAT). The UN reports indicated above have comprehensive references and sources covering, 
inter alia, such omissions. There are other important UN reports, for example from UNICEF. 


Cases and examples from Italy, with its enormous Cultural Heritage, should offer considerable 
lessons internationally, but the writer's knowledge of the Italian scene is limited, so responses 
to the following types of questions will be invited at EVA Florence 2015: 


- Current status and directions in Italy? 


- Exemplars of leading practice? 
- Possibilities of direct assistance to other countries? 
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It is hoped to deepen and extend this initial discussion paper and pursue a dialogue on these 
issues later in 2015, including at EVA St Petersburg (June) and EVA London (July). 
Improvements are still needed — including in the *developed countries'-- as shown for 
example by Lisney et al. [7] 


References 


[1] The Universal Declaration of Human Rights. United Nations General Assembly in Paris 
on 10 December 1948 General Assembly resolution 217 A. http://www.un.org/en/universal- 
declaration-human-rights. 


[2] United Nations Convention on the Rights of Persons with Disabilities [UN CPRD] 2006. 
http://www.un.org/disabilities/convention/conventionfull.shtml. 


[3] World Report on Disability, World Health Organization & World Bank , 2011 
http://www.who. int/disabilities/world_report/201 1/report.pdf. 


[4] UNESCO Global Report Opening New Avenues for Empowerment ICTs to Access 
Information and Knowledge for Persons with Disabilities, PWD, 2013. 
http://unesdoc.unesco.org/images/002 1/002197/219767e.pdf 


[5] European Disability Strategy 2010-2020: A Renewed Commitment to a Barrier-Free 
Europe 2010. European Commission. http://eur- 
lex.europa.eu/LexUriServ/LexUriServ.do?uri-COM:2010:0636:FIN:EN:PDF 


[6] Proposal for a DIRECTIVE OF THE EUROPEAN PARLIAMENT AND OF THE 
COUNCIL on the approximation of the laws, regulations and administrative provisions of the 
Member States as regards the accessibility requirements for products and services 
(2/12/2015) http://ec.europa.eu/social/main.jsp?langId-en&catId-89&newsId-2400 


[7] Lisney, Eleanor; Bowen, Jonathan P; Hearn, Kirsten and Zedda, Maria. Museums and 
Technology: Being Inclusive Helps Accessibility for All. CURATOR: THE MUSEUM 
JOURNAL. Volume 56, Issue 3, July 2013, P353-361. 


101 


TOVIVA PROJECT: DOCUMENTING THE SPANISH 
DEFENSE TOWERS ALONG THE VALENCIAN COAST 
WITH A COMPREHENSIVE DIGITAL METHODOLOGY 


A. Pablo Rodríguez-Navarro 
Departamento de Expresión Gráfica Arquitectónica 
Univesitat Politécnica de Valéncian 
Valencia, Spain. 
rodriguez@upv.es 
B. Giorgio Verdiani 
Dipartimento di Architettura 
Universita degli Studi di Firenze 
Firenze, Italy. 
giorgio.verdiani@unifi.it 
C. Teresa Gil Piqueras 
Departamento de Expresión Gráfica Arquitectónica 
Univesitat Politécnica de Valéncian 
Valencia, Spain. 


tgil@ega.upv.es 


Abstract — The Spanish Mediterranean coasts, offers a suggestive sequence of towers, they 
were built, since the sixteenth century, to protect the territory from pirates. Once defeated the 
piracy of that time in the “Mare Nostrum”, the coastal towers system lost its defensive function, 
but remained as a testimony of history. The Toviva project, supported by the Spanish Ministry 
and developed by a Spanish-Italian team, is aimed to the creation of a virtual network using 
advanced digital tools, where the towers become components for knowledge dissemination and a 
link between the cultural heritage of a territory and its history. 


INTRODUCTION 


Spain, in its Mediterranean coasts, offers a suggestive sequence of towers, this system is 
integrated and showed in the landscape for defence and surveillance purposes. It has 
represented an articulated military system since the XVI century, when there was the need to 
protect a territory otherwise at risk of being attacked by pirates. Once the pirates were 
defeated in the “Mare Nostrum", the coastal towers system lost its defensive function, but 
remained as a historical witness. Such a heritage can now benefit from a specific digital 
approach. In this, as under development in the research presented here it is important to 
analyze what are the final research needs, while they rise a list of products to be obtained and 
will result from the point of view of the graphical documentation, in a number of plans to 
scale, photogrammetries, 3D models, pictures and videos [1]. The first step is defining an 
appropriate methodology which may include traditional direct surveys, photogrammetric 
surveys (2D correction, 3D restitution), 3D laser scanning, UAV/drones aerial shooting, 
photographs, spherical panoramas, videos, etc... The Toviva project, supported by the Spanish 
Ministry and developed by a Spanish-Italian team, is aimed to contribute in creating a virtual 
network of towers as an amplification of the real buildings, where the towers become 
component of dissemination of knowledge and represent a link between the cultural heritage 
of a territory and its history. 


Historical notes 


In 1453 Constantinople was conquered by Sultan Mehmed II. This event signed the 
beginning of the fight to control the Mediterranean Coast. The quantity of pirate attacks 
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augmented and this caused the conquest of Argel in 1516 by Arrouj Barbarrosa. During the 
XVI century pirate attacks increased and expanded around the coast causing sacking and 
robbery [2]. This century was also a period of confluence as never seen before along the 
Valencian Coast. A complex social-political context and the technological development 
produced the birth of a very articulated military architecture system in the territory. In 1557, 
Felipe II became King and began an ambitious project which was previously traced by the 
Duke of Maqueda, it was based on building a network of watchtowers, fortresses, walls and 
defences to protect the coast. Giovanni Battista Antonelli "the elder" was entrusted for the 
realization of this ambitious project. He was followed by other members of his own family. In 
his ideas about defence it is possible to see the comparing of the coast with a wall, where 
towns were the doors, the villages were the bastions and watchtowers were the battlements. 


Specific features of the Valencian Coast 

The Valencian coast is characterized by mixing rough areas with sandy lands and lightly 
slopes. The locations of every tower followed tactical criteria, getting the higher view of the 
coast and controlling landing for attack or obtaining provisions. However, the higher points of 
flat lands were used to get the capacity of viewing the sea and towards the horizon [3]. 

In 1554, *Las ordenanzas de la Guardia Marítima del reino de Valencia" was published to 
organize the defensive system of Valencian coast and to describe the army organization that 
every tower needed: Guards were responsible of looking after the coast and warning in case of 
danger, and “Atajadores” were responsible of communicating news between towers. This 
strategic network spread around all the Mediterranean area, especially and with clear 
architectural “system” in all the areas controlled or influenced by the Spanish Reign. 


VALENCIAN TOWERS: TYPOLOGIES 


The typologies of towers along the Valencian coast have gone developing in time and they 
have been adapted to new necessities and technological developments of attack solutions and 
strategies. From the architectural point of view, building typologies changed their 
morphology throughout XV and XVI centuries. It is possible to recognize 3 main stages [3]: 
Stage 1: From the end of the XV century to the beginning of the XVI century. Towers have 
prismatic shape with squared or circular base, the structure is very simple, answering to the 
need of visual control of the territory. Stage 2: From 1520 to 1560, the towers are pre- 
bastioned, they have singular elements like alambores and esperontes (sloping and screeding 
systems). Stage 3: XVII century, towers have important variations regarding previous phases. 
The use of conical shapes allows a higher resistance against artillery. The hexagonal shape 
appeared widening the eyesight and enhancing the offensive functions. 

Towers were composed by three parts, foundation, base and main body. Each part had a 
structural and defensive function. The diameter of foundation generally was bigger than the 
tower base and it was adapted to the terrain. Base reaches a third of the total height and its 
function was to avoid the access and to reject artillery shots. The main part includes watching 
area while the interiors were built using vaulted system. The specific elements of these 
typologies were: an elevated access by a ladder, holes on the shaft, loopholes, machicolations. 
All these elements gave to the tower the necessary support to functions for defending and 
controlling the surrounding areas. 


DIGITAL SURVEY AND 3D MODELS POST PROCESSING 
The first objective of the Toviva project is to provide a methodology for the graphic 


documentation of coastal tower architectures, where the generation of proper metadata linked 
to 3D models is a task empowering the possibilities of interpretation and effective sharing. 
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The Toviva project proposes the use of advanced technical tools for graphical 
representation, employing a flexible methodology, trying to simplify processes and 
optimizing resources while maintaining the maximum reliability and quality of results. In 
addition, these same results can be adapted for cultural uses of a general public. 

The first inspection is established after obtaining the following basic items: Location: town 
and access to the tower (conditions and issues). / Cartography of the area. / Ownership: 
public or private. Needed Permissions. / In case of building with a current occupation: contact 
managers; Tourist Office, dealership / Contact with local technicians and scholars: 
archaeologist, architect, engineer / Conditions for the access to the interior of the tower. 

During the first inspection a set of photos and sketches is realized. The photographs are 
taken according to a specific procedure, enriched by notes and setting the Exif data in proper 
ways, so that later on it is easier to work on these archives. A specific record of the structure 
is is prepared, with the following main information: 

Location: On plain or mountain / Exempt or within an urban area / Visible/accessible or 
not around its entire perimeter. 

Form: Polygonal / Rounded / Flown elements / Specific Shape / 

Dimensions: Height / Number of floors / Number of rooms per floor / 

Staircase: Size / Shape - Coverage/roof; accessible or not. 

Inside Presence of movable objects - Presence of annexed buildings. 

Presence of trees and/or invasive vegetation - Other features. 


Ground and UAV Photogrammetry 

After the remarkable evolution of the recent years, photogrammetry and UAV/Drone 
survey became a significant tool for architecture survey. For photogrammetry this “rebirth” is 
due to new algorithms that have facilitated the production of 3D models using S.f.M. 
(Structure from Motion) method [4]. When working with this kind of software, the workflow 
starts from the alignment of the pictures. The procedure is based on the use of each pixel of 
the image as a point, looking for homologous points in all the pictures and thereby the relative 
positions of each camera. The set of camera positions, along with the pictures themselves, are 
used in the following phase, with the construction of the model’s geometry. In the end it is 
possible to create a photo-realistic texture for the final 3D model, ready to be exported to 
other software to be used for analysis, drawing production, multimedia. 

Using UAV unit (drone) for taking aerial photographs solves many accessibility problems. 
Aerial photographs allow a great freedom in taking pictures from any point of view. I is 
possible to use the same camera for ground shooting or to prefer different cameras and then 
mixing the images with no significant variation in the workflow. But in general, with a 
double sequence of pictures, one from the ground, one from the air, it is possible to reduce the 
post processing needs. In example, none of the pictures taken from the ground need the 
masking of the sky process: the presence of aerial pictures defines clearly the borders of a 
tower, allowing a perfect processing without noisy artefacts in the skyline of the architecture. 


3D Laser Scanner 

For complex structures, especially in presence of articulated internal/external parts, the use 
of 3D laser scanner technology is necessary. It solves the problem posed by photo-modeling 
for interiors, whether for lighting reasons or for a question of space and complexity. Scanning 
is much more laborious, both for data acquisition and subsequent processing (alignment of the 
point-clouds, mesh generation, surfaces treatments, texturing, etc...), but when correctly 
operated, it ensures excellent data acquisition under any circumstances, keeping the quality of 
all the data gathered with the same accuracy [4,5]. A specific topographic support is operated 
for the 3D Laser Scanner survey, it is limited to the targets applied this survey, reducing 
needs for overlapping scans and making easier and robust the process of alignment of isolated 
rooms and of the coverage/roof. 
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A Multimedia Catalogue 

The Toviva project is based on the use of 3D photo-realistic models with the inclusion of 
their full data, including each model in its geographic location to facilitate their individuation 
and support the dissemination task. The opportunity of the project is to expand the visibility 
of these cultural assets along the Valencian coast, providing data to the general public through 
a “light” computing platforms such as Smart-Phones or PDAs (using APPs for virtual and 
augmented reality). The access will be through the Internet, using local WiFi network or the 
telephone data access. The strategies are based on the use of easily accesible solutions, like 
the Sketchfab (www.sketchfab.com) platform, while for more complex and site specific 
situation it is possible to foreseen more complex and articulated solutions, based on 
multimedia/programming software. The will is to create a versatile standard, capable to be 
easily replied in other experiences in the future, but with a solid and complete archive in its 
background. The catalogue, now open to the public in its online version 
(http://toviva.blogs.upv.es/) even if still in ongoing development, shows the system of 
defensive elements as a tool of knowledge and tourism guide for the Valencian Mediterranean 
coast, a solution usable by scholars, researchers, students, tourists, curious and any kind of 
interested user. The contents are organized around graphical elements like pictures and (when 
possible) digital survey derived representations. This system will connect to the visitor 
position useful information about the territory, the history of the tower and the system of the 
fortifications. At the same time, an annual conference, the Fortmed (www.fortmed.eu), helps 
in connecting scholars and researcher on this subject. 


Conclusions 

The watchtowers located in along the Mediterranean coast are historical witnesses of a 
defensive system from the XVI and XVII centuries. The goal to use them as a base to transmit 
knowledge through advanced data collection finds important solutions in the digital tools for 
survey and representation, thus, such an archive will be quite limited if not shared. And this 
can be supported by interactive schemes and 3D visualizations. Multimedia information will 
allow to enhance the awareness about the systems of towers, to better understand how each 
single tower is part of a system along the Mediterranean coast, created for defending against 
Barbary attacks but still capable to drive a network of knowledge between past and present. 
At the same time, an accurate reconstruction of buildings is a significant step in the 
understanding of the “evolution” of this Heritage. All the previous documents, collected and 
organized, are the base for a correct evaluation. Proper technologies allow to explain the 
defensive system along Mediterranean coast connecting new experiences to consolidated 
knowledge. Multimedia tools may help people to take a look in a distant past, not only to 
obtain an idea about tower systems with high historical and patrimonial interest but also to 
better understand the complex system of relationships spreading all across the Mediterranean. 
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Fig. 2 - Towers typologies in the set analysed by Toviva Project, (R.Atzeni, V. Naldini [6]) 


Fig. 3 - 3D Laser Scanner and Octacopter Drone at work (G. Verdiani, S. Giraudeau) 


Fig. 4 - 3D model with texturing exported in sketchfab.com (P. Rodriguez-Navarro) 
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Fig. 5 - The Fortmed2016 and the Toviva Websites (P. Rodriguez-Navarro, G. Verdiani) 
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Abstract 


This paper presents a pioneering experiment in digital art carried out in Brazil, UK, 
USA since the years 1970. An account of the origins, development of the 
achievements of methods, disciples and partners in the advance of the use of 
technology in art and art historic legacy. An analysis of the overall action philosophy 
identifies the convergence with the renaissance ideals of Verrochio’s Bottega. A 
summary of main works commissioned by corporate and collector clients is presented 
with considerations on future activities and overall market trends. The overall 
framework of the atelier Murat has been used as a pattern to implement similar 
organizations as a consulting service. Multiple Mecenas and Clients are exploring the 
possibility of placing large scale order and have them fulfilled in very short time and 
vey high quality. Among them Builders, Hotels, Hospitals, Universities and large 
corporations have already become regular partners of this promising experiment in 
ECOLOGICAL DIGITAL ART FOR CORPORATE INTERIOR DECORATION 


and can be visualized in the following blog: 


http://artesvisuaisdemurat.blogspot.com.br/ 
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